Documentation
¶
Overview ¶
A REST api for the artifact store, implemented using Martini.
Each "Handle" function acts as a handler for a request and is routed with Martini (routing is hanlded elsewhere).
Index ¶
- Constants
- func CloseArtifact(ctx context.Context, artifact *model.Artifact, db database.Database, ...) error
- func CloseBucket(ctx context.Context, bucket *model.Bucket, db database.Database, ...) error
- func GetArtifactContent(ctx context.Context, r render.Render, req *http.Request, ...)
- func GetArtifactContentChunks(ctx context.Context, r render.Render, req *http.Request, ...)
- func HandleCloseArtifact(ctx context.Context, r render.Render, db database.Database, ...)
- func HandleCloseBucket(ctx context.Context, r render.Render, db database.Database, ...)
- func HandleCreateArtifact(ctx context.Context, r render.Render, req *http.Request, db database.Database, ...)
- func HandleCreateBucket(ctx context.Context, r render.Render, req *http.Request, db database.Database, ...)
- func HandleGetArtifact(ctx context.Context, r render.Render, artifact *model.Artifact)
- func HandleGetBucket(ctx context.Context, r render.Render, bucket *model.Bucket)
- func ListArtifacts(ctx context.Context, r render.Render, req *http.Request, db database.Database, ...)
- func ListBuckets(ctx context.Context, r render.Render, db database.Database)
- func LogAndRespondWithError(ctx context.Context, render render.Render, code int, err error)
- func LogAndRespondWithErrorf(ctx context.Context, render render.Render, code int, errStr string, ...)
- func MergeLogChunks(ctx context.Context, artifact *model.Artifact, db database.Database, ...) error
- func PostArtifact(ctx context.Context, r render.Render, req *http.Request, db database.Database, ...)
- func PutArtifact(ctx context.Context, artifact *model.Artifact, db database.Database, ...) error
- func RespondWithError(ctx context.Context, render render.Render, code int, err error)
- func RespondWithErrorf(ctx context.Context, render render.Render, code int, errStr string, ...)
- type HttpError
- func AppendLogChunk(ctx context.Context, db database.Database, artifact *model.Artifact, ...) *HttpError
- func CreateArtifact(req createArtifactReq, bucket *model.Bucket, db database.Database) (*model.Artifact, *HttpError)
- func CreateBucket(db database.Database, clk common.Clock, bucketId string, owner string) (*model.Bucket, *HttpError)
- func NewHttpError(code int, format string, args ...interface{}) *HttpError
- func NewWrappedHttpError(code int, err error) *HttpError
- type PutArtifactReq
Constants ¶
const DEFAULT_DEADLINE = 30
const DuplicateArtifactNameFormat = "%s.dup.%s"
Format string to construct unique artifact names to recover from duplicate artifact name.
const MaxArtifactSizeBytes = 200 * 1024 * 1024
Maximum artifact size => 200 MB
const MaxChunkedRequestBytes = 1000000
Maximum number of bytes to fetch while returning chunked response.
const MaxDuplicateFileNameResolutionAttempts = 5
Maximum number of duplicate file name resolution attempts before failing with an internal error.
const MaxUploadAttempts = 3
MaxUploadAttempts is the maximum number of attempts to upload an artifact to S3.
const ReadaheadBytes = 1 * 1024 * 1024
Number of bytes to read ahead while making DB queries.
Variables ¶
This section is empty.
Functions ¶
func CloseArtifact ¶
func CloseArtifact(ctx context.Context, artifact *model.Artifact, db database.Database, s3bucket *s3.Bucket, failIfAlreadyClosed bool) error
CloseArtifact closes an artifact for further writes and begins process of merging and uploading the artifact. This operation is only valid for artifacts which are being uploaded in chunks. In all other cases, an error is returned.
func CloseBucket ¶
func CloseBucket(ctx context.Context, bucket *model.Bucket, db database.Database, s3Bucket *s3.Bucket, clk common.Clock) error
CloseBucket closes a bucket, preventing further updates. All artifacts associated with the bucket are also marked closed. If the bucket is already closed, an error is returned.
func GetArtifactContent ¶
func GetArtifactContentChunks ¶
func GetArtifactContentChunks(ctx context.Context, r render.Render, req *http.Request, res http.ResponseWriter, db database.Database, s3bucket *s3.Bucket, artifact *model.Artifact)
GetArtifactContentChunks lists artifact contents in a chunked form. Useful to poll for updates to chunked artifacts. All artifact types are supported and chunks can be requested from arbitrary locations within artifacts.
This is primarily meant for Changes UI for log following. If you need to fetch byte ranges from the store, it should be available directly at /content
URL query parameters offset and limit can be used to control range of chunks to be fetched. offset -> byte offset of the start of the range to be fetched (defaults to beginning of artifact) limit -> number of bytes to be fetched (defaults to 100KB)
Negative values for any query parameter will cause it to be set to 0 (default)
func HandleCloseArtifact ¶
func HandleCloseArtifact(ctx context.Context, r render.Render, db database.Database, s3bucket *s3.Bucket, artifact *model.Artifact)
HandleCloseArtifact handles the HTTP request to close an artifact. See CloseArtifact for details.
func HandleCloseBucket ¶
func HandleCloseBucket(ctx context.Context, r render.Render, db database.Database, bucket *model.Bucket, s3Bucket *s3.Bucket, clk common.Clock)
HandleCloseBucket handles the HTTP request to close a bucket. See CloseBucket for details.
func HandleCreateArtifact ¶
func HandleCreateBucket ¶
func HandleGetArtifact ¶
func HandleGetBucket ¶
func ListArtifacts ¶
func LogAndRespondWithError ¶
LogAndRespondWithError posts a JSON-serialized error and statuscode on the HTTP response object (using Martini render). Log the error message to Sentry.
func LogAndRespondWithErrorf ¶
func LogAndRespondWithErrorf(ctx context.Context, render render.Render, code int, errStr string, params ...interface{})
LogAndRespondWithErrorf posts a JSON-serialized error message and statuscode on the HTTP response object (using Martini render). Log the error message to Sentry.
func MergeLogChunks ¶
func MergeLogChunks(ctx context.Context, artifact *model.Artifact, db database.Database, s3bucket *s3.Bucket) error
Merges all of the individual chunks into a single object and stores it on s3. The log chunks are stored in the database, while the object is uploaded to s3.
func PostArtifact ¶
func PostArtifact(ctx context.Context, r render.Render, req *http.Request, db database.Database, s3bucket *s3.Bucket, artifact *model.Artifact)
PostArtifact updates content associated with an artifact.
If the artifact is streamed (uploaded in one shot), PutArtifact is invoked to stream content directly through to S3.
If the artifact is chunked (appended chunk by chunk), verify that the position being written to matches the current end of artifact, insert a new log chunk at that position and move the end of file forward.
func PutArtifact ¶
func PutArtifact(ctx context.Context, artifact *model.Artifact, db database.Database, bucket *s3.Bucket, req PutArtifactReq) error
PutArtifact writes a streamed artifact to S3. The entire file contents are streamed directly through to S3. If S3 is not accessible, we don't make any attempt to buffer on disk and fail immediately.
func RespondWithError ¶
RespondWithError posts a JSON-serialized error and statuscode on the HTTP response object (using Martini render).
Types ¶
type HttpError ¶
type HttpError struct {
// contains filtered or unexported fields
}
func AppendLogChunk ¶
func AppendLogChunk(ctx context.Context, db database.Database, artifact *model.Artifact, logChunkReq *createLogChunkReq) *HttpError
AppendLogChunk appends a logchunk to an artifact. If the logchunk position does not match the current end of artifact, an error is returned. An exception to this is made when the last seen logchunk is repeated, which is silently ignored without an error.
func CreateArtifact ¶
func CreateArtifact(req createArtifactReq, bucket *model.Bucket, db database.Database) (*model.Artifact, *HttpError)
CreateArtifact creates a new artifact in a open bucket.
If an artifact with the same name already exists in the same bucket, we attempt to rename the artifact by adding a suffix. If the request specifies a chunked artifact, the size field is ignored and always set to zero. If the request is for a streamed artifact, size is mandatory. A relative path field may be specified to preserve the original file name and path. If no path is specified, the original artifact name is used by default.
Source Files