taskcluster-worker: github.com/taskcluster/taskcluster-worker/runtime/fetcher Index | Files

package fetcher

import "github.com/taskcluster/taskcluster-worker/runtime/fetcher"

Package fetcher provides means for plugins and engines to fetch resources with generic references. Hence, the format for reference is consistent across plugins and engines, and we can re-use the download logic.


Package Files

combine.go doc.go fetcher.go filereseter.go urlfetcher.go

type Context Uses

type Context interface {
    context.Context      // Context for aborting the fetch operation
    Queue() client.Queue // Client with credentials covering Fetcher.Scopes()
    // Print a progress report that looks somewhat like this:
    //     "Fetching <description> - <percent> %"
    // Progress reports won't be sent more than once every 10 seconds.
    Progress(description string, percent float64)

Context for fetching resource from a reference.

type Fetcher Uses

type Fetcher interface {
    // Schema for references, should **only** match this type
    Schema() schematypes.Schema

    // Unique key for a reference (reference must validate against schema)
    // Useful for caching resources.
    HashKey(reference interface{}) string

    // List of scope-sets that could grant access to the reference.
    // Tasks using a cached instance of this resources should satisfy at-least
    // one of these scope-sets.
    // Returns [][]string{[]string{}} if no scopes are required.
    Scopes(reference interface{}) [][]string

    // Fetch a reference to a target, sending progress to Context as well
    // as returning a human readable error message, if fetching fails.
    Fetch(context Context, reference interface{}, target WriteSeekReseter) error

A Fetcher specifies a schema for references that it knows how to fetch. It also provides a method to generate a HashKey for each valid reference, as well as a list of scopes required for a task to use a reference.

var URL Fetcher = urlFetcher{}

URL is Fetcher for downloading files from a URL.

func Combine Uses

func Combine(fetchers ...Fetcher) Fetcher

Combine a list of Fetchers into a single Fetcher implementation.

For this to work well, the reference schemas should all be distinct such that no reference matches more than one fetcher. If there is ambiguity the first Fetcher whose schema matches will be used.

type FileReseter Uses

type FileReseter struct {

FileReseter implements WriteSeekReseter for an *os.File instance

func (*FileReseter) Reset Uses

func (f *FileReseter) Reset() error

Reset will truncate the file and seek to the beginning.

type WriteSeekReseter Uses

type WriteSeekReseter interface {
    Reset() error

WriteSeekReseter is a io.Writer + io.Seeker + io.Closer with Reset() method that discards everything written and starts over from scratch.

This is easily implemented by wrapping os.File with FileReseter.

Package fetcher imports 11 packages (graph). Updated 2017-03-18. Refresh now. Tools for package owners.