pkgsite: golang.org/x/pkgsite/internal/fetch Index | Files

package fetch

import "golang.org/x/pkgsite/internal/fetch"

Package fetch provides a way to fetch modules from a proxy.

Package fetch provides a way to fetch modules from a proxy.

Package fetch provides a way to fetch modules from a proxy.

Package fetch provides a way to fetch modules from a proxy.

Index

Package Files

fetch.go fetchlocal.go limit.go load.go loadshedding.go package.go readme.go unit.go

Constants

const (

    // MaxFileSize is the maximum filesize that is allowed for reading.
    // The fetch process should fail if it encounters a file exceeding
    // this limit.
    MaxFileSize = 30 * megabyte
)

Limits for discovery worker.

Variables

var (

    // FetchLatencyDistribution aggregates frontend fetch request
    // latency by status code. It does not count shedded requests.
    FetchLatencyDistribution = &view.View{
        Name:        "go-discovery/worker/fetch-latency",
        Measure:     fetchLatency,
        Aggregation: ochttp.DefaultLatencyDistribution,
        Description: "Fetch latency by result status.",
        TagKeys:     []tag.Key{dcensus.KeyStatus},
    }
    // FetchResponseCount counts fetch responses by status.
    FetchResponseCount = &view.View{
        Name:        "go-discovery/worker/fetch-count",
        Measure:     fetchLatency,
        Aggregation: view.Count(),
        Description: "Fetch request count by result status",
        TagKeys:     []tag.Key{dcensus.KeyStatus},
    }
    // FetchPackageCount counts how many packages were successfully fetched.
    FetchPackageCount = &view.View{
        Name:        "go-discovery/worker/fetch-package-count",
        Measure:     fetchedPackages,
        Aggregation: view.Count(),
        Description: "Count of packages successfully fetched",
    }
    // SheddedFetchCount counts the number of fetches that were shedded.
    SheddedFetchCount = &view.View{
        Name:        "go-discovery/worker/fetch-shedded",
        Measure:     fetchesShedded,
        Aggregation: view.Count(),
        Description: "Count of shedded fetches",
    }
)
var (
    LocalVersion    = "latest"
    LocalCommitTime = time.Time{}
)

Version and commit time are pre specified when fetching a local module, as these fields are normally obtained from a proxy.

type BadPackageError Uses

type BadPackageError struct {
    Err error // Not nil.
}

BadPackageError represents an error loading a package because its contents do not make up a valid package.

This can happen, for example, if the .go files fail to parse or declare different package names.

func (*BadPackageError) Error Uses

func (bpe *BadPackageError) Error() string

type FetchInfo Uses

type FetchInfo struct {
    ModulePath string
    Version    string
    ZipSize    uint64
    Start      time.Time
    Finish     time.Time
    Status     int
    Error      error
}

func FetchInfos Uses

func FetchInfos() []*FetchInfo

FetchInfos returns information about all fetches in progress, sorted by start time.

type FetchResult Uses

type FetchResult struct {
    ModulePath           string
    RequestedVersion     string
    ResolvedVersion      string
    GoModPath            string
    Status               int
    Error                error
    Defer                func() // caller must defer this on all code paths
    Module               *internal.Module
    PackageVersionStates []*internal.PackageVersionState
}

func FetchLocalModule Uses

func FetchLocalModule(ctx context.Context, modulePath, localPath string, sourceClient *source.Client) *FetchResult

FetchLocalModule fetches a module from a local directory and process its contents to return an internal.Module and other related information. modulePath is not necessary if the module has a go.mod file, but if both exist, then they must match. FetchResult.Error should be checked to verify that the fetch succeeded. Even if the error is non-nil the result may contain useful data.

func FetchModule Uses

func FetchModule(ctx context.Context, modulePath, requestedVersion string, proxyClient *proxy.Client, sourceClient *source.Client) (fr *FetchResult)

FetchModule queries the proxy or the Go repo for the requested module version, downloads the module zip, and processes the contents to return an *internal.Module and related information.

Even if err is non-nil, the result may contain useful information, like the go.mod path.

Callers of FetchModule must

defer fr.Defer()

immediately after the call.

type LoadShedStats Uses

type LoadShedStats struct {
    SizeInFlight     uint64
    MaxSizeInFlight  uint64
    RequestsInFlight int
    RequestsShed     int
    RequestsTotal    int
}

LoadShedStats holds statistics about load shedding.

func ZipLoadShedStats Uses

func ZipLoadShedStats() LoadShedStats

ZipLoadShedStats returns a snapshot of the current LoadShedStats for zip files.

Package fetch imports 42 packages (graph) and is imported by 8 packages. Updated 2020-10-24. Refresh now. Tools for package owners.