stats

package
v2.1.2+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 13, 2019 License: Apache-2.0 Imports: 8 Imported by: 1

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type APTBucketReaderStats 

type APTBucketReaderStats struct {
	InstitutionsCached      []*models.Institution
	WorkItemsCached         []*models.WorkItem
	WorkItemsFetched        []*models.WorkItem
	WorkItemsCreated        []*models.WorkItem
	WorkItemsQueued         []*models.WorkItem
	WorkItemsMarkedAsQueued []*models.WorkItem
	S3Items                 []string
	Errors                  []string
	Warnings                []string
}

APTBucketReaderStats records stats for the apt_bucket_reader worker. It's not meant for regular production, because it can use a lot of member when the bucket reader encounters thousands of objects. It's primarily useful for integration tests when we want the bucket reader to tell us what it did, so we can compare that to what actually happened. It's also useful periodicall in production, to diagnose problems in smaller runs.

func APTBucketReaderStatsLoadFromFile 

func APTBucketReaderStatsLoadFromFile(pathToFile string) (*APTBucketReaderStats, error)

Loads bucket reader stats from a JSON file like the ones that APTBucketReaderStats.DumpToFile dumps out.

func NewAPTBucketReaderStats 

func NewAPTBucketReaderStats() *APTBucketReaderStats

Creates a new, empty APTBucketReaderStats object.

func (*APTBucketReaderStats) AddError 

func (stats *APTBucketReaderStats) AddError(message string)

Adds an error message to the stats.

func (*APTBucketReaderStats) AddS3Item 

func (stats *APTBucketReaderStats) AddS3Item(bucketAndKey string)

Adds an item to the list of files that the bucket reader found in the S3 receiving buckets. Param bucketAndKey should be something like "aptrust.receiving.virginia.edu/virginia.edu_12345678.tar"

func (*APTBucketReaderStats) AddToInstitutionsCached 

func (stats *APTBucketReaderStats) AddToInstitutionsCached(inst *models.Institution)

Adds an institution to the list of cached institutions.

func (*APTBucketReaderStats) AddWarning 

func (stats *APTBucketReaderStats) AddWarning(message string)

Adds a warning to the stats.

func (*APTBucketReaderStats) AddWorkItem 

func (stats *APTBucketReaderStats) AddWorkItem(listName string, item *models.WorkItem) error

Adds a WorkItem to a list.

func (*APTBucketReaderStats) DumpToFile 

func (stats *APTBucketReaderStats) DumpToFile(pathToFile string) error

Dumps a JSON representation of this object to a file at the specified path. This will overwrite the existing file, if the existing file has a .json extension. Note that converting the stats object to JSON can use a lot of memory, if you're working with a lot of data. This is safe for integration testing, and it dumps out human-readable formatted JSON. See also APTBucketReaderStatsLoadFromFile.

func (*APTBucketReaderStats) FindWorkItemById 

func (stats *APTBucketReaderStats) FindWorkItemById(listName string, id int) (*models.WorkItem, error)

Returns the WorkItem with the matching ID, or nil.

func (*APTBucketReaderStats) FindWorkItemByNameAndEtag 

func (stats *APTBucketReaderStats) FindWorkItemByNameAndEtag(listName, name, etag string) (*models.WorkItem, error)

Does what it says.

func (*APTBucketReaderStats) HasErrors 

func (stats *APTBucketReaderStats) HasErrors() bool

Returns true if this object contains any errors

func (*APTBucketReaderStats) HasWarnings 

func (stats *APTBucketReaderStats) HasWarnings() bool

Returns true if this object contains any warnings

func (*APTBucketReaderStats) InstitutionByIdentifier 

func (stats *APTBucketReaderStats) InstitutionByIdentifier(identifier string) *models.Institution

Finds an Institution in the cache by identifier. Returns nil if not found.

func (*APTBucketReaderStats) InstitutionsCachedContains 

func (stats *APTBucketReaderStats) InstitutionsCachedContains(identifier string) bool

Returns true if the Institution with the specified identifier is in the Institutions cache.

func (*APTBucketReaderStats) S3ItemWasFound 

func (stats *APTBucketReaderStats) S3ItemWasFound(bucketAndKey string) bool

Returns true if the specified bucketAndKey was found in S3

type APTQueueStats 

type APTQueueStats struct {
	ItemsQueued         map[string][]*models.WorkItem
	ItemsMarkedAsQueued []*models.WorkItem
	Errors              []string
	Warnings            []string
}

APTQueueStats records information about what apt_queue did.

func APTQueueStatsLoadFromFile 

func APTQueueStatsLoadFromFile(pathToFile string) (*APTQueueStats, error)

APTQueueStatsLoadFromFile loads APTQueueStats from a JSON file.

func NewAPTQueueStats 

func NewAPTQueueStats() *APTQueueStats

NewAPTQueueStats creates a new, empty APTQueueStats object.

func (*APTQueueStats) AddError 

func (stats *APTQueueStats) AddError(message string)

Adds an error message to the stats.

func (*APTQueueStats) AddItemMarkedAsQueued 

func (stats *APTQueueStats) AddItemMarkedAsQueued(item *models.WorkItem)

AddItemMarkedAsQueued adds a WorkItem to the items marked as queued. Marked means apt_queue told Pharos that the item has been added to the appropriate NSQ topic.

func (*APTQueueStats) AddWarning 

func (stats *APTQueueStats) AddWarning(message string)

Adds a warning to the stats.

func (*APTQueueStats) AddWorkItem 

func (stats *APTQueueStats) AddWorkItem(topic string, item *models.WorkItem)

AddWorkItem adds a WorkItem to the specified topic. This is to record the fact that apt_queue added the specified WorkItem id to the specified NSQ topic.

func (*APTQueueStats) DumpToFile 

func (stats *APTQueueStats) DumpToFile(pathToFile string) error

DumpToFile dumps a JSON representation of this object to a file at the specified path. This will overwrite the existing file, if the existing file has a .json extension. Note that converting the stats object to JSON can use a lot of memory, if you're working with a lot of data. This is safe for integration testing, and it dumps out human-readable formatted JSON. See also APTQueueStatsLoadFromFile.

func (*APTQueueStats) FindMarkedItemByName 

func (stats *APTQueueStats) FindMarkedItemByName(name string) *models.WorkItem

FindMarkedItemByName returns the "marked as queued" item with the specified name. Name is the bag name as it appears in the receiving bucket, with the tar extension, e.g. "my_bag.tar".

func (*APTQueueStats) FindQueuedItemByName 

func (stats *APTQueueStats) FindQueuedItemByName(name string) (*models.WorkItem, string)

FindQueuedItemByName returns the queued WorkItem with the specified name, or nil. It also returns the name of the topic the WorkItem was added to. Name is the bag name as it appears in the receiving bucket, with the tar extension, e.g. "my_bag.tar".

func (*APTQueueStats) HasErrors 

func (stats *APTQueueStats) HasErrors() bool

Returns true if this object contains any errors

func (*APTQueueStats) HasWarnings 

func (stats *APTQueueStats) HasWarnings() bool

Returns true if this object contains any warnings

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.