metax

package
v0.10.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 28, 2020 License: GPL-3.0 Imports: 18 Imported by: 0

Documentation

Overview

Package metax provides a client for the CSC MetaX API.

Index

Constants

View Source 
const (
	Verbose = false

	DatasetsEndpoint                = "/rest/datasets/"
	ChangeCumulativeStateEndpoint   = "/rpc/datasets/change_cumulative_state"
	RefreshDirectoryContentEndpoint = "/rpc/datasets/refresh_directory_content"
)
View Source 
const (
	SchemaIda = "metax-ida"
	SchemaAtt = "metax-att"
)

These constants are used to identify the schema a dataset is using. When creating a new dataset, the schema will also determine the catalog the dataset belongs to.

View Source 
const (
	// MetaxDatasetFamily is the dataset type for Fairdata datasets.
	MetaxDatasetFamily = 2
)

Variables

View Source 
var (
	ErrInvalidContentType = errors.New("invalid content-type: expected json")
	ErrNotFound           = errors.New("not found")
	ErrIdRequired         = errors.New("dataset without id and not allowed to create")
	ErrInvalidId          = errors.New("invalid dataset id")
)
View Source 
var (
	// IdentifierKey is the key pointing to the primary identifier, if any.
	IdentifierKey = "identifier"

	// NewVersionKey is the key to the value indicating a new version has been created.
	NewVersionKey = "new_version_created"

	// NewVersionId is the key holding the identifier of the new version or an empty string.
	NewVersionIdKey = "new_version_created.identifier"

	// Editor is the key pointing to Qvain's private object inside the dataset.
	EditorKey = "editor"

	// QvainId is the key with the Qvain record id.
	QvainIdKey = "record_id"

	// QvainId is the key with the identifier "qvain".
	QvainIdentifierKey = "identifier"

	// DateCreatedKey is the key for the Metax dataset creation timestamp.
	DateCreatedKey = "date_created"

	// DateModifiedKey is the key for the Metax dataset modification timestamp.
	DateModifiedKey = "date_modified"

	// DateDeprecatedKey is the key for the Metax dataset deprecation timestamp.
	DateDeprecatedKey = "date_deprecated"

	// DateRemovedKey is the key for the Metax dataset removal timestamp.
	DateRemovedKey = "date_removed"
)
View Source 
var (
	// CatalogIdentifiers are used to identify schema in datasets imported from Metax.
	CatalogIdentifiers = map[string]string{
		"urn:nbn:fi:att:data-catalog-ida": SchemaIda,
		"urn:nbn:fi:att:data-catalog-att": SchemaAtt,
		"urn:nbn:fi:att:data-catalog-pas": SchemaIda,
	}
)

Functions

func CreatedNewVersion 

func CreatedNewVersion(blob []byte) bool

CreatedNewVersion returns a boolean indicating whether the new version created key exists. Note this doesn't check if its value is valid.

func DisableHttps 

func DisableHttps(svc *MetaxService)

func GetIdentifier 

func GetIdentifier(blob []byte) string

func GetIdentifiers 

func GetIdentifiers(blob []byte) (string, string, string)

GetIdentifiers returns the Metax identifier, Metax "new version" identifier and Qvain record_id fields.

func GetModificationDate 

func GetModificationDate(blob []byte) time.Time

GetModificationDate returns the date of last modification (created, modified, deprecated, removed) for a dataset.

func GetQvainId 

func GetQvainId(blob []byte) string

GetQvainId returns the Qvain id of the dataset from the editor object or empty string.

func IsPublished 

func IsPublished(blob []byte) bool

func LoadMetaxDataset 

func LoadMetaxDataset(ds *models.Dataset) models.TypedDataset

LoadMetaxDataset constructs an existing MetaxDataset from an existing base dataset.

func MaybeNewVersionId 

func MaybeNewVersionId(blob []byte) string

MaybeNewVersionId returns the id of a new Metax version if a new version was created, otherwise an empty string.

func NewMetaxDataset 

func NewMetaxDataset(creator uuid.UUID) (models.TypedDataset, error)

NewMetaxDataset creates a Metax dataset.

func WithLatestVersion 

func WithLatestVersion(svc *MetaxService)

func WithStreaming 

func WithStreaming(req *http.Request)

WithStreaming forces the API to return a streaming response without pagination.

Types

type ApiError 

type ApiError struct {
	// contains filtered or unexported fields
}

func (*ApiError) Error 

func (e *ApiError) Error() string

func (*ApiError) OriginalError 

func (e *ApiError) OriginalError() json.RawMessage

func (*ApiError) StatusCode 

func (e *ApiError) StatusCode() int

type DataCatalog 

type DataCatalog struct {
	Identifier *string `json:"identifier"`
}

DataCatalog contains the catalog identifier

type DatasetOption 

type DatasetOption func(*http.Request)

func Since 

func Since(t time.Time) DatasetOption

Since is a dataset option that adds an if-modified-since header to dataset queries.

func WithOwner 

func WithOwner(uid string) DatasetOption

WithOwner is a dataset option that restricts dataset queries to the Qvain owner set in the dataset's editor object.

func WithRemoved 

func WithRemoved() DatasetOption

WithRemoved is a dataset option that filters dataset queries to those that have been removed.

func WithUser 

func WithUser(id string) DatasetOption

WithUser is a dataset option that filters dataset queries to those that have the given user identifier set in the dataset.

type Editor 

type Editor struct {
	Identifier *string `json:"identifier"`
	RecordId   *string `json:"record_id"`
	CreatorId  *string `json:"creator_id,omitempty"`
	OwnerId    *string `json:"owner_id,omitempty"`
	ExtId      *string `json:"fd_id,omitempty"`
}

Editor is the Go representation of the Editor object in a Metax record.

type LinkingError 

type LinkingError struct {
	// contains filtered or unexported fields
}

LinkingError is a custom error type that adds the missing field name.

func NewLinkingError 

func NewLinkingError(field ...string) *LinkingError

NewLinkingError creates a new LinkingError with the field name set to the argument, or marks the whole metadata block as missing if no argument given.

func (*LinkingError) Error 

func (e *LinkingError) Error() string

Error satisfies the Error interface.

func (*LinkingError) Field 

func (e *LinkingError) Field() string

Field returns the missing field name.

func (*LinkingError) IsNotMine 

func (e *LinkingError) IsNotMine() bool

IsNotMine returns a boolean value indicating if the whole Qvain metadata block was missing.

type MetaxDataset 

type MetaxDataset struct {
	*models.Dataset
}

MetaxDataset wraps a models.Dataset.

func (*MetaxDataset) CreateData 

func (dataset *MetaxDataset) CreateData(family int, schema string, blob []byte, extra map[string]string) error

CreateData creates a dataset from template and merges set fields.

func (*MetaxDataset) UpdateData 

func (dataset *MetaxDataset) UpdateData(family int, schema string, blob []byte, extra map[string]string) error

UpdateData creates a partial dataset JSON blob to patch an existing one with.

func (*MetaxDataset) ValidateCreated added in v0.10.3 

func (dataset *MetaxDataset) ValidateCreated() error

ValidateCreated performs checks on created dataset.

func (*MetaxDataset) ValidateUpdated added in v0.10.3 

func (dataset *MetaxDataset) ValidateUpdated(updated *models.Dataset) error

ValidateUpdated checks that updated dataset can be saved.

type MetaxOption 

type MetaxOption func(*MetaxService)

func WithCredentials 

func WithCredentials(user, pass string) MetaxOption

func WithInsecureCertificates 

func WithInsecureCertificates(enable bool) MetaxOption

func WithLogger 

func WithLogger(logger zerolog.Logger) MetaxOption

func WithUserAgent 

func WithUserAgent(ua string) MetaxOption

type MetaxRawRecord 

type MetaxRawRecord struct {
	json.RawMessage
}

MetaxRawRecord embeds a json.RawMessage containing an unparsed JSON []byte slice with the Metax dataset.

func (MetaxRawRecord) GetQvainId 

func (raw MetaxRawRecord) GetQvainId(mrec *MetaxRecord) (*uuid.UUID, error)

IsNewDataset checks if the Metax record should be treated as a new dataset. A dataset is new if it doesn't have a Qvain id.

func (MetaxRawRecord) Record 

func (raw MetaxRawRecord) Record() (*MetaxRecord, error)

Record unmarshals the raw JSON and validates it, returning either a partially parsed MetaxRecord or an error.

-wvh- NOTE: (2019-03-28) Validation disabled to allow creating new datasets.

func (MetaxRawRecord) ToQvain 

func (raw MetaxRawRecord) ToQvain() (*models.Dataset, bool, error)

ToQvain converts a Metax record in raw JSON to a Qvain record using the values in the Editor object.

If the Editor metadata contains valid data, consider the dataset ours and populate (all) the Dataset struct fields; boolean New is false. If the Editor metadata does not exist, consider the dataset new and let the caller handle creation; the ids and ownership fields are not set. If the Editor metadata is invalid, return an error.

type MetaxRecord 

type MetaxRecord struct {
	Id         int64  `json:"id"`
	Identifier string `json:"identifier"`

	DataCatalog *DataCatalog `json:"data_catalog"`

	MetadataProviderUser *string `json:"metadata_provider_user"`

	DateCreated  *time.Time `json:"date_created"`
	DateModified *time.Time `json:"date_modified"`

	Removed bool `json:"removed"`

	Editor *Editor `json:"editor"`

	ResearchDataset json.RawMessage `json:"research_dataset"`
	Contract        json.RawMessage `json:"contract"`
}

MetaxRecord is a helper struct to parse the fields we need from a Metax dataset.

func (*MetaxRecord) Validate 

func (record *MetaxRecord) Validate() error

Validate checks if the Metax record contains the fields we need to identify the record (those below the `editor` key).

-wvh- NOTE: (2019-03-28) Deprecated to allow creating new datasets if there is no existing application metadata in the dataset.

type MetaxService 

type MetaxService struct {
	// contains filtered or unexported fields
}

MetaxService represents the Metax API server.

func NewMetaxService 

func NewMetaxService(host string, params ...MetaxOption) *MetaxService

NewMetaxService returns a Metax API client.

func (*MetaxService) ChangeCumulativeState added in v0.10.3 

func (api *MetaxService) ChangeCumulativeState(ctx context.Context, identifier string, cumulativeState string) (newMetaxId string, err error)

ChangeCumulativeState calls Metax RPC for changing cumulative state of a dataset with given Metax identifier. Returns the new Metax identifier if a new dataset version was created.

func (*MetaxService) Create 

func (api *MetaxService) Create(ctx context.Context, blob json.RawMessage) (json.RawMessage, error)

Create makes new datasets at the API endpoint. Deprecated: use Store().

func (*MetaxService) Datasets 

func (api *MetaxService) Datasets(params ...DatasetOption) (*PaginatedResponse, error)

func (*MetaxService) Delete 

func (api *MetaxService) Delete(ctx context.Context, blob json.RawMessage) error

Delete marks a dataset as removed in Metax.

func (*MetaxService) GetId 

func (api *MetaxService) GetId(id string) (json.RawMessage, error)

GetId queries the dataset endpoint for a dataset with the given id.

func (*MetaxService) GetIdRemoved 

func (api *MetaxService) GetIdRemoved(id string) (json.RawMessage, error)

GetId queries the dataset endpoint for a dataset with the given id.

func (*MetaxService) ReadStream deprecated 

func (api *MetaxService) ReadStream(params ...DatasetOption) ([]MetaxRecord, error)

ReadStream queries the dataset endpoint with an unpaged request.

Deprecated: use ReadStreamChannel() for actual asynchronous stream processing.

func (*MetaxService) ReadStreamChannel 

func (api *MetaxService) ReadStreamChannel(ctx context.Context, params ...DatasetOption) (int, chan *MetaxRawRecord, chan error, error)

ReadStreamChannel queries the dataset endpoint streaming the resulting datasets asynchronously throught a channel.

func (*MetaxService) RefreshDirectoryContent added in v0.10.3 

func (api *MetaxService) RefreshDirectoryContent(ctx context.Context,
	datasetIdentifier string, directoryIdentifier string) (newMetaxId string, err error)

RefreshDirectoryContent calls Metax RPC for updating directory contents on a published dataset. If the directory has new files that aren't yet in the dataset, they will be added. Returns the new Metax identifier if a new dataset version was created.

func (*MetaxService) Store 

func (api *MetaxService) Store(ctx context.Context, blob json.RawMessage, owner *models.User) (json.RawMessage, error)

Store sends – or "publishes" – a dataset to the Metax dataset API. If the dataset has no identifier yet, it is POSTed to the dataset endpoint as a new dataset; otherwise it is PUT to the endpoint for that specific dataset identifier. If the request was successful, the dataset will be returned; if the request failed, and the response was in JSON format, the API error will include that error body.

func (*MetaxService) UrlForId 

func (api *MetaxService) UrlForId(id string) string

type PaginatedResponse 

type PaginatedResponse struct {
	Count    int            `json:"count"`
	Next     string         `json:"next"`
	Previous string         `json:"previous"`
	Results  []*MetaxRecord `json:"results"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.