engines

package
v1.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 5, 2020 License: Apache-2.0 Imports: 38 Imported by: 0

Documentation

Overview

Package engines provides the Reverse Proxy, Reverse Proxy Cache, and Time Series Delta Proxy Cache features for use by handlers.

Index

Constants

View Source 
const (
	// RevalStatusNone indicates the object will not undergo revalidation against the origin
	RevalStatusNone = RevalidationStatus(iota)
	// RevalStatusInProgress indicates the object is currently being revalidated against the origin
	RevalStatusInProgress
	// RevalStatusLocal is used during a cache Range Miss, and indicates that while the user-requested ranges
	// for an object are uncached and must be fetched from the origin, other ranges for the object are cached
	// but require revalidation. When a request is in this state, Trickster will use the response headers from
	// the range miss to locally revalidate the cached content instead of making a separate request to the
	// origin for revalidating the cached ranges.
	RevalStatusLocal
	// RevalStatusOK indicates the object was successfully revalidated against the origin and is still fresh
	RevalStatusOK
	// RevalStatusFailed indicates the origin returned a new object for the URL to replace the cached version
	RevalStatusFailed
)
View Source 
const HTTPBlockSize = 32 * 1024

HTTPBlockSize represents 32K of bytes

Variables

View Source 
var ErrReadIndexTooLarge = errors.New("Read index too large")

ErrReadIndexTooLarge is an error indicating the read index is too large

View Source 
var Reqs sync.Map

Reqs is for Progressive Collapsed Forwarding

Functions

func CheckIfNoneMatch 

func CheckIfNoneMatch(etag string, headerValue string, ls status.LookupStatus) bool

CheckIfNoneMatch determines if the provided match value satisfies an "If-None-Match" condition against the cached object. As Trickster is a cache, matching is always weak.

func DeltaProxyCacheRequest 

func DeltaProxyCacheRequest(w http.ResponseWriter, r *http.Request)

DeltaProxyCacheRequest identifies the gaps between the cache and a new timeseries request, requests the gaps from the origin server and returns the reconstituted dataset to the downstream request while caching the results for subsequent requests of the same data

func DoProxy 

func DoProxy(w io.Writer, r *http.Request) *http.Response

DoProxy proxies an inbound request to its corresponding upstream origin with no caching features

func FetchViaObjectProxyCache 

func FetchViaObjectProxyCache(r *http.Request) ([]byte, *http.Response, bool)

FetchViaObjectProxyCache Fetches an object from Cache or Origin (on miss), writes the object to the cache, and returns the object to the caller

func ObjectProxyCacheRequest 

func ObjectProxyCacheRequest(w http.ResponseWriter, r *http.Request)

ObjectProxyCacheRequest provides a Basic HTTP Reverse Proxy/Cache

func PrepareFetchReader 

func PrepareFetchReader(r *http.Request) (io.ReadCloser, *http.Response, int64)

PrepareFetchReader prepares an http response and returns io.ReadCloser to provide the response data, the response object and the content length. Used in Fetch.

func PrepareResponseWriter 

func PrepareResponseWriter(w io.Writer, code int, header http.Header) io.Writer

PrepareResponseWriter prepares a response and returns an io.Writer for the data to be written to. Used in Respond.

func Respond 

func Respond(w http.ResponseWriter, code int, header http.Header, body []byte)

Respond sends an HTTP Response down to the requesting client

func WriteCache 

func WriteCache(ctx context.Context, c cache.Cache, key string, d *HTTPDocument, ttl time.Duration, compressTypes map[string]bool) error

WriteCache writes an HTTPDocument to the cache

Types

type CachingPolicy 

type CachingPolicy struct {
	IsFresh           bool      `msg:"is_fresh"`
	NoCache           bool      `msg:"nocache"`
	NoTransform       bool      `msg:"notransform"`
	FreshnessLifetime int       `msg:"freshness_lifetime"`
	CanRevalidate     bool      `msg:"can_revalidate"`
	MustRevalidate    bool      `msg:"must_revalidate"`
	LastModified      time.Time `msg:"last_modified"`
	Expires           time.Time `msg:"expires"`
	Date              time.Time `msg:"date"`
	LocalDate         time.Time `msg:"local_date"`
	ETag              string    `msg:"etag"`

	IsNegativeCache bool `msg:"is_negative_cache"`

	IfNoneMatchValue      string    `msg:"-"`
	IfModifiedSinceTime   time.Time `msg:"-"`
	IfUnmodifiedSinceTime time.Time `msg:"-"`

	IsClientConditional  bool `msg:"-"`
	IsClientFresh        bool `msg:"-"`
	HasIfModifiedSince   bool `msg:"-"`
	HasIfUnmodifiedSince bool `msg:"-"`
	HasIfNoneMatch       bool `msg:"-"`

	IfNoneMatchResult bool `msg:"-"`
}

CachingPolicy defines the attributes for determining the cachability of an HTTP object

func GetRequestCachingPolicy 

func GetRequestCachingPolicy(h http.Header) *CachingPolicy

GetRequestCachingPolicy examines HTTP request headers for caching headers and true if the corresponding response is OK to cache

func GetResponseCachingPolicy 

func GetResponseCachingPolicy(code int, negativeCache map[int]time.Duration, h http.Header) *CachingPolicy

GetResponseCachingPolicy examines HTTP response headers for caching headers a returns a CachingPolicy reference

func (*CachingPolicy) Clone 

func (cp *CachingPolicy) Clone() *CachingPolicy

Clone returns an exact copy of the Caching Policy

func (*CachingPolicy) DecodeMsg 

func (z *CachingPolicy) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*CachingPolicy) EncodeMsg 

func (z *CachingPolicy) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*CachingPolicy) MarshalMsg 

func (z *CachingPolicy) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*CachingPolicy) Merge 

func (cp *CachingPolicy) Merge(src *CachingPolicy)

Merge merges the source CachingPolicy into the subject CachingPolicy

func (*CachingPolicy) Msgsize 

func (z *CachingPolicy) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*CachingPolicy) ParseClientConditionals 

func (cp *CachingPolicy) ParseClientConditionals()

ParseClientConditionals inspects the client http request to determine if it includes any conditions

func (*CachingPolicy) ResetClientConditionals 

func (cp *CachingPolicy) ResetClientConditionals()

ResetClientConditionals sets the request-specific conditional values of the subject caching policy to false, so as to facilitate reuse of the policy with subsequent requests for the same cache object

func (*CachingPolicy) ResolveClientConditionals 

func (cp *CachingPolicy) ResolveClientConditionals(ls status.LookupStatus)

ResolveClientConditionals ensures any client conditionals are handled before responding to the client request

func (*CachingPolicy) String 

func (cp *CachingPolicy) String() string

func (*CachingPolicy) TTL 

func (cp *CachingPolicy) TTL(multiplier float64, max time.Duration) time.Duration

TTL returns a TTL based on the subject caching policy and the provided multiplier and max values

func (*CachingPolicy) UnmarshalMsg 

func (z *CachingPolicy) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type HTTPDocument 

type HTTPDocument struct {
	StatusCode    int                 `msg:"status_code"`
	Status        string              `msg:"status"`
	Headers       map[string][]string `msg:"headers"`
	Body          []byte              `msg:"body"`
	ContentLength int64               `msg:"content_length"`
	ContentType   string              `msg:"content_type"`
	CachingPolicy *CachingPolicy      `msg:"caching_policy"`
	// Ranges is the list of Byte Ranges contained in the body of this document
	Ranges     byterange.Ranges              `msg:"ranges"`
	RangeParts byterange.MultipartByteRanges `msg:"-"`
	// StoredRangeParts is a version of RangeParts that can be exported to MessagePack
	StoredRangeParts map[string]*byterange.MultipartByteRange `msg:"range_parts"`
	// contains filtered or unexported fields
}

HTTPDocument represents a full HTTP Response/Cache Document with unbuffered body

func DocumentFromHTTPResponse 

func DocumentFromHTTPResponse(resp *http.Response, body []byte, cp *CachingPolicy) *HTTPDocument

DocumentFromHTTPResponse returns an HTTPDocument from the provided HTTP Response and Body

func QueryCache 

func QueryCache(ctx context.Context, c cache.Cache, key string, ranges byterange.Ranges) (*HTTPDocument, status.LookupStatus, byterange.Ranges, error)

QueryCache queries the cache for an HTTPDocument and returns it

func (*HTTPDocument) DecodeMsg 

func (z *HTTPDocument) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*HTTPDocument) EncodeMsg 

func (z *HTTPDocument) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*HTTPDocument) FulfillContentBody 

func (d *HTTPDocument) FulfillContentBody() error

FulfillContentBody will concatenate the document's Range parts into a single, full content body the caller must know that document's multipart ranges include full content length before calling this method

func (*HTTPDocument) LoadRangeParts 

func (d *HTTPDocument) LoadRangeParts()

LoadRangeParts convert a StoredRangeParts into a RangeParts

func (*HTTPDocument) MarshalMsg 

func (z *HTTPDocument) MarshalMsg(b []byte) (o []byte, err error)

MarshalMsg implements msgp.Marshaler

func (*HTTPDocument) Msgsize 

func (z *HTTPDocument) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*HTTPDocument) ParsePartialContentBody 

func (d *HTTPDocument) ParsePartialContentBody(resp *http.Response, body []byte)

ParsePartialContentBody parses a Partial Content response body into 0 or more discrete parts

func (*HTTPDocument) SetBody 

func (d *HTTPDocument) SetBody(body []byte)

SetBody sets the Document Body as well as the Content Length, based on the length of body. This assumes that the caller has checked that the request is not a Range request

func (*HTTPDocument) Size 

func (d *HTTPDocument) Size() int

Size returns the size of the HTTPDocument's headers, CachingPolicy, RangeParts, Body and timeseries data

func (*HTTPDocument) UnmarshalMsg 

func (z *HTTPDocument) UnmarshalMsg(bts []byte) (o []byte, err error)

UnmarshalMsg implements msgp.Unmarshaler

type IndexReader 

type IndexReader func(index uint64, b []byte) (int, error)

IndexReader implements a reader to read data at a specific index into slice b

type ProgressiveCollapseForwarder 

type ProgressiveCollapseForwarder interface {
	AddClient(io.Writer) error
	Write([]byte) (int, error)
	Close()
	IndexRead(uint64, []byte) (int, error)
	WaitServerComplete()
	WaitAllComplete()
	GetBody() ([]byte, error)
	GetResp() *http.Response
}

ProgressiveCollapseForwarder accepts data written through the io.Writer interface, caches it and makes all the data written available to n readers. The readers can request data at index i, to which the PCF may block or return the data immediately.

func NewPCF 

func NewPCF(resp *http.Response, contentLength int64) ProgressiveCollapseForwarder

NewPCF returns a new instance of a ProgressiveCollapseForwarder

type RevalidationStatus 

type RevalidationStatus int

RevalidationStatus enumerates the methodologies for maintaining time series cache data

func (RevalidationStatus) String 

func (s RevalidationStatus) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.