luci: Index | Files | Directories

package internal

import ""


Package Files

checksum.go instancecache.go tagcache.go


var ErrUnknownSHA256 = errors.New("no sha256 is recorded in the file")

ErrUnknownSHA256 indicates the deserialized message doesn't have SHA256 set.

This can happen when deserializing records in the old format.

func MarshalWithSHA256 Uses

func MarshalWithSHA256(pm proto.Message) ([]byte, error)

MarshalWithSHA256 serializes proto message to bytes, calculates SHA256 checksum of it, and returns serialized envelope that contains both.

UnmarshalWithSHA256 can then be used to verify SHA256 and deserialized the original object.

func UnmarshalWithSHA256 Uses

func UnmarshalWithSHA256(buf []byte, pm proto.Message) error

UnmarshalWithSHA256 is reverse of MarshalWithSHA256.

It checks SHA256 checksum and deserializes the object if it matches the blob.

If the expected SHA256 is not available in 'buf', returns ErrUnknownSHA256. This can happen when reading blobs in old format that used SHA1.

type InstanceCache Uses

type InstanceCache struct {
    // contains filtered or unexported fields

InstanceCache is a file-system-based, thread-safe, LRU cache of instances.

Does not validate instance hashes; it is caller's responsibility.

func NewInstanceCache Uses

func NewInstanceCache(fs fs.FileSystem) *InstanceCache

NewInstanceCache initializes InstanceCache.

fs will be the root of the cache.

func (*InstanceCache) GC Uses

func (c *InstanceCache) GC(ctx context.Context, now time.Time)

GC opportunistically purges entries that haven't been touched for too long.

func (*InstanceCache) Get Uses

func (c *InstanceCache) Get(ctx context.Context, pin common.Pin, now time.Time) (pkg.Source, error)

Get searches for the instance in the cache and opens it for reading.

If the instance is not found, returns an os.IsNotExists error.

func (*InstanceCache) Put Uses

func (c *InstanceCache) Put(ctx context.Context, pin common.Pin, now time.Time, write func(*os.File) error) error

Put caches an instance.

write must write the instance contents. May remove some instances from the cache that were not accessed for a long time.

type TagCache Uses

type TagCache struct {
    // contains filtered or unexported fields

TagCache provides a mapping (package name, tag) -> instance ID.

This mapping is safe to cache because tags are not detachable: once a tag is successfully resolved to an instance ID it is guaranteed to resolve to same instance ID later or not resolve at all (e.g. if one tag is attached to multiple instances, in which case the tag is misused anyway). In any case, returning a cached instance ID does make sense. The primary purpose of this cache is to avoid round trips to the service to increase reliability of 'cipd ensure' calls that use only tags to specify versions. It happens to be the most common case of 'cipd ensure' usage by far.

Additionally, this TagCache stores a mapping of (pin, file_name) -> encode(ObjectRef of extracted file) to assist in the `selfupdate` flow.

Whenever selfupdate resolves what CIPD package instance ID (pin) it SHOULD be at, it looks at '(pin, 'cipd') => binary hash' map to figure out what hash the client itself SHOULD have for this instance ID. The client then calculates the hash of itself to see if it's actually already at that instance ID.

func NewTagCache Uses

func NewTagCache(fs fs.FileSystem, service string) *TagCache

NewTagCache initializes TagCache.

fs will be the root of the cache. It will be searched for tagcache.db file.

func (*TagCache) AddExtractedObjectRef Uses

func (c *TagCache) AddExtractedObjectRef(ctx context.Context, pin common.Pin, fileName string, ref *api.ObjectRef) error

AddExtractedObjectRef records that fileName extracted from the package at the given pin has the given hash.

The hash is represented as ObjectRef, which is a tuple (hash algo, hex digest).

Call 'Save' later to persist these changes to the cache file on disk.

func (*TagCache) AddTag Uses

func (c *TagCache) AddTag(ctx context.Context, pin common.Pin, tag string) error

AddTag records that (pin.PackageName, tag) maps to pin.InstanceID.

Call 'Save' later to persist these changes to the cache file on disk.

func (*TagCache) ResolveExtractedObjectRef Uses

func (c *TagCache) ResolveExtractedObjectRef(ctx context.Context, pin common.Pin, fileName string) (*api.ObjectRef, error)

ResolveExtractedObjectRef returns ObjectRef or nil if that file is not in the cache.

Returns error if the cache can't be read.

func (*TagCache) ResolveTag Uses

func (c *TagCache) ResolveTag(ctx context.Context, pkg, tag string) (pin common.Pin, err error)

ResolveTag returns cached tag or empty Pin{} if such tag is not in the cache.

Returns error if the cache can't be read.

func (*TagCache) Save Uses

func (c *TagCache) Save(ctx context.Context) error

Save stores all pending cache updates to the file system.

It effectively resets the object to the initial state.


retryPackage retry contains helpers for doing tight retry loops.

Package internal imports 22 packages (graph) and is imported by 2 packages. Updated 2021-01-24. Refresh now. Tools for package owners.