README
¶
[![GoDoc Widget]][GoDoc]
Spyglass
A spyglass is an lensed monocular maritime instrument used to see things that may have been difficult to see otherwise.
Spyglass is a pluggable artifact viewer framework for Prow and a crude metaphor for the real object. It collects artifacts (usually files in a storage bucket) from various sources and distributes them to registered viewers, which are responsible for consuming them and rendering a view.
A typical Spyglass page might look something like this:
A general Spyglass query will proceed as follows:
- User provides a job source in the query (usually a job name and build ID).
- Spyglass finds all artifact names associated with the given job source.
- Spyglass builds a cache of which artifacts match which lenses via configured regular expressions.
- Lenses with matching artifacts are pre-rendered in order of descending priority.
- Spyglass then sends render requests to each registered lens with its matching artifacts.
- Each lens performs any necessary operations on the artifacts and produces a blob of HTML.
- Views (HTML) are inserted asynchronously as viewers return.
Lenses
A lens is an set of functions that consume a list of artifacts and produces some HTML.
Lens names are unique, and must much the package name for the lens.
Built-in Viewers
Spyglass comes with some built-in viewers for commonly produced artifacts.
- Prow Metadata
Name: metadata Title: Metadata Match: finished.json|started.json Priority: 0 - JUnit
Name: junit Title: JUnit Matches: artifacts/junit.*\.xml Priority: 5 - Logs
Name: buildlog Title: Build Log Matches: build-log.txt|pod-log Priority: 10
Building your own viewer
Building a viewer consists of three main steps.
Write Boilerplate
First, create a package lensnamehere under prow/spyglass/lenses and
import the lenses package.
Implement
Next, implement the necessary functions for a viewer. More specifically, implement the following interface (defined in lenses.go):
type Lens interface {
// Name returns the name of your lens (which must match the name of the directory it lives in)
Name() string
// Title returns a human-readable title for your lens.
Title() string
// Priority returns a number that is used to determine the ordering of your lens (lower is more important)
Priority() int
// Header is used to inject content into the lens's <head>. It will only ever be called once per load.
Header(artifacts []Artifact, resourceDir string) string
// Body is used to generate the contents of the lens's <body>. It will initially be called with empty data, but
// the lens front-end code may choose to re-render itself with custom data.
Body(artifacts []Artifact, resourceDir string, data string) string
// Callback is used for the viewer to exchange arbitrary data with the frontend. It is called with lens-specified
// data, and returns data to be passed to the lens. JSON encoding is recommended in both directions.
Callback(artifacts []Artifact, resourceDir string, data string) string
}
In the init method, call lenses.RegisterLens() with an instance of your implementation of the interface.
Spyglass should now be aware of your lens.
Additionally, some front-end TypeScript code can be provided. Configure your BUILD.bazel to build it, then emit a
Documentation
¶
Overview ¶
Package spyglass creates views for Prow job artifacts.
Index ¶
- Variables
- type GCSArtifact
- func (a *GCSArtifact) CanonicalLink() string
- func (a *GCSArtifact) JobPath() string
- func (a *GCSArtifact) ReadAll() ([]byte, error)
- func (a *GCSArtifact) ReadAt(p []byte, off int64) (n int, err error)
- func (a *GCSArtifact) ReadAtMost(n int64) ([]byte, error)
- func (a *GCSArtifact) ReadTail(n int64) ([]byte, error)
- func (a *GCSArtifact) Size() (int64, error)
- type GCSArtifactFetcher
- type LensRequest
- type PodLogArtifact
- func (a *PodLogArtifact) CanonicalLink() string
- func (a *PodLogArtifact) JobPath() string
- func (a *PodLogArtifact) ReadAll() ([]byte, error)
- func (a *PodLogArtifact) ReadAt(p []byte, off int64) (n int, err error)
- func (a *PodLogArtifact) ReadAtMost(n int64) ([]byte, error)
- func (a *PodLogArtifact) ReadTail(n int64) ([]byte, error)
- func (a *PodLogArtifact) Size() (int64, error)
- type PodLogArtifactFetcher
- type Spyglass
- func (s *Spyglass) FetchArtifacts(src string, podName string, sizeLimit int64, artifactNames []string) ([]lenses.Artifact, error)
- func (s *Spyglass) JobPath(src string) (string, error)
- func (s *Spyglass) Lenses(matchCache map[string][]string) []lenses.Lens
- func (s *Spyglass) ListArtifacts(src string) ([]string, error)
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCannotParseSource is returned by newGCSJobSource when an incorrectly formatted source string is passed ErrCannotParseSource = errors.New("could not create job source from provided source") )
Functions ¶
This section is empty.
Types ¶
type GCSArtifact ¶
type GCSArtifact struct {
// contains filtered or unexported fields
}
GCSArtifact represents some output of a prow job stored in GCS
func NewGCSArtifact ¶
func NewGCSArtifact(ctx context.Context, handle artifactHandle, link string, path string, sizeLimit int64) *GCSArtifact
NewGCSArtifact returns a new GCSArtifact with a given handle, canonical link, and path within the job
func (*GCSArtifact) CanonicalLink ¶
func (a *GCSArtifact) CanonicalLink() string
CanonicalLink gets the GCS web address of the artifact
func (*GCSArtifact) JobPath ¶
func (a *GCSArtifact) JobPath() string
JobPath gets the GCS path of the artifact within the current job
func (*GCSArtifact) ReadAll ¶
func (a *GCSArtifact) ReadAll() ([]byte, error)
ReadAll will either read the entire file or throw an error if file size is too big
func (*GCSArtifact) ReadAt ¶
func (a *GCSArtifact) ReadAt(p []byte, off int64) (n int, err error)
ReadAt reads len(p) bytes from a file in GCS at offset off
func (*GCSArtifact) ReadAtMost ¶
func (a *GCSArtifact) ReadAtMost(n int64) ([]byte, error)
ReadAtMost reads at most n bytes from a file in GCS. If the file is compressed (gzip) in GCS, n bytes of gzipped content will be downloaded and decompressed into potentially GREATER than n bytes of content.
func (*GCSArtifact) ReadTail ¶
func (a *GCSArtifact) ReadTail(n int64) ([]byte, error)
ReadTail reads the last n bytes from a file in GCS
func (*GCSArtifact) Size ¶
func (a *GCSArtifact) Size() (int64, error)
Size returns the size of the artifact in GCS
type GCSArtifactFetcher ¶
type GCSArtifactFetcher struct {
// contains filtered or unexported fields
}
GCSArtifactFetcher contains information used for fetching artifacts from GCS
func NewGCSArtifactFetcher ¶
func NewGCSArtifactFetcher(c *storage.Client) *GCSArtifactFetcher
NewGCSArtifactFetcher creates a new ArtifactFetcher with a real GCS Client
type LensRequest ¶
LensRequest holds data sent by a view
type PodLogArtifact ¶
type PodLogArtifact struct {
// contains filtered or unexported fields
}
PodLogArtifact holds data for reading from a specific pod log
func NewPodLogArtifact ¶
func NewPodLogArtifact(jobName string, buildID string, sizeLimit int64, ja jobAgent) (*PodLogArtifact, error)
NewPodLogArtifact creates a new PodLogArtifact
func (*PodLogArtifact) CanonicalLink ¶
func (a *PodLogArtifact) CanonicalLink() string
CanonicalLink returns a link to where pod logs are streamed
func (*PodLogArtifact) JobPath ¶
func (a *PodLogArtifact) JobPath() string
JobPath gets the path within the job for the pod log. Always returns build-log.txt. This is because the pod log becomes the build log after the job artifact uploads are complete, which should be used instead of the pod log.
func (*PodLogArtifact) ReadAll ¶
func (a *PodLogArtifact) ReadAll() ([]byte, error)
ReadAll reads all available pod logs, failing if they are too large
func (*PodLogArtifact) ReadAt ¶
func (a *PodLogArtifact) ReadAt(p []byte, off int64) (n int, err error)
ReadAt implements reading a range of bytes from the pod logs endpoint
func (*PodLogArtifact) ReadAtMost ¶
func (a *PodLogArtifact) ReadAtMost(n int64) ([]byte, error)
ReadAtMost reads at most n bytes
func (*PodLogArtifact) ReadTail ¶
func (a *PodLogArtifact) ReadTail(n int64) ([]byte, error)
ReadTail reads the last n bytes of the pod log
func (*PodLogArtifact) Size ¶
func (a *PodLogArtifact) Size() (int64, error)
Size gets the size of the pod log. Note: this function makes the same network call as reading the entire file.
type PodLogArtifactFetcher ¶
type PodLogArtifactFetcher struct {
// contains filtered or unexported fields
}
PodLogArtifactFetcher is used to fetch artifacts from k8s apiserver
func NewPodLogArtifactFetcher ¶
func NewPodLogArtifactFetcher(ja jobAgent) *PodLogArtifactFetcher
NewPodLogArtifactFetcher returns a PodLogArtifactFetcher using the given job agent as storage
type Spyglass ¶
type Spyglass struct {
// JobAgent contains information about the current jobs in deck
JobAgent *jobs.JobAgent
// ConfigAgent contains information about the prow configuration
ConfigAgent configAgent
*GCSArtifactFetcher
*PodLogArtifactFetcher
}
Spyglass records which sets of artifacts need views for a Prow job. The metaphor can be understood as follows: A spyglass receives light from a source through an eyepiece, which has a lens that ultimately presents a view of the light source to the observer. Spyglass receives light (artifacts) via a source (src) through the eyepiece (Eyepiece) and presents the view (what you see in your browser) via a lens (Lens).
func (*Spyglass) FetchArtifacts ¶
func (s *Spyglass) FetchArtifacts(src string, podName string, sizeLimit int64, artifactNames []string) ([]lenses.Artifact, error)
FetchArtifacts constructs and returns Artifact objects for each artifact name in the list. This includes getting any handles needed for read write operations, direct artifact links, etc.
Source Files
¶
Directories
¶
| Path | Synopsis |
|---|---|
|
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers
|
Package lenses provides interfaces and methods necessary for implementing custom artifact viewers |
|
buildlog
Package buildlog provides a build log viewer for Spyglass
|
Package buildlog provides a build log viewer for Spyglass |
|
junit
Package junit provides a junit viewer for Spyglass
|
Package junit provides a junit viewer for Spyglass |
|
metadata
Package metadata provides a metadata viewer for Spyglass
|
Package metadata provides a metadata viewer for Spyglass |