queue

package
v0.0.0-...-b8abe08 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 5, 2024 License: BSD-3-Clause Imports: 6 Imported by: 0

Documentation

Overview

Package queue provides a queue interface that can be used for asynchronous scheduling of fetch actions.

Index

Constants

View Source
const (
	DisableProxyFetchParam = "proxyfetch"
	DisableProxyFetchValue = "off"
	SourceParam            = "source"
	SourceFrontendValue    = "frontend"
	SourceWorkerValue      = "worker"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type InMemory

type InMemory struct {
	// contains filtered or unexported fields
}

InMemory is a Queue implementation that schedules in-process fetch operations. Unlike the GCP task queue, it will not automatically retry tasks on failure.

This should only be used for local development.

func NewInMemory

func NewInMemory(ctx context.Context, workerCount int, experiments []string, processFunc InMemoryProcessFunc) *InMemory

NewInMemory creates a new InMemory that asynchronously fetches from proxyClient and stores in db. It uses workerCount parallelism to execute these fetches.

func (*InMemory) ScheduleFetch

func (q *InMemory) ScheduleFetch(ctx context.Context, modulePath, version string, _ *Options) (bool, error)

ScheduleFetch pushes a fetch task into the local queue to be processed asynchronously.

func (*InMemory) WaitForTesting

func (q *InMemory) WaitForTesting(ctx context.Context)

WaitForTesting waits for all queued requests to finish. It should only be used by test code.

type InMemoryProcessFunc

type InMemoryProcessFunc func(context.Context, string, string) (int, error)

type Options

type Options struct {
	// DisableProxyFetch reports whether proxyfetch should be set to off when
	// making a fetch request.
	DisableProxyFetch bool

	// Suffix is used to force reprocessing of tasks that would normally be
	// de-duplicated. It is appended to the task name.
	Suffix string

	// Source is the source that requested the task to be queued. It is
	// either "frontend" or the empty string if it is the worker.
	Source string
}

Options is used to provide option arguments for a task queue.

type Queue

type Queue interface {
	ScheduleFetch(ctx context.Context, modulePath, version string, opts *Options) (bool, error)
}

A Queue provides an interface for asynchronous scheduling of fetch actions.

Directories

Path Synopsis
Package gcpqueue provides a GCP queue implementation that can be used for asynchronous scheduling of fetch actions.
Package gcpqueue provides a GCP queue implementation that can be used for asynchronous scheduling of fetch actions.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL