storage

package
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 5, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	GSStorageName = "gs"
)

GSStorageName is a Google Storage name.

View Source 
const (
	LocalStorageName = "file"
)

LocalStorageName is a file storage name.

View Source 
const (
	S3StorageName = "s3"
)

S3StorageName is a s3 storage name.

Variables

This section is empty.

Functions

func ExtractBucketAndPrefix added in v0.4.0 

func ExtractBucketAndPrefix(uri string) (string, string, error)

ExtractBucketAndPrefix extracts bucket name and prefix path from provided uri. uri could be any S3 compatible storage uri like AWS S3 or Google Storage. after processing of uri s3://fasttrackml/2/30357ed2eaac4f2cacdbcd0e06e9e48a/artifacts result will be: - bucket = fasttrackml - prefix = 2/30357ed2eaac4f2cacdbcd0e06e9e48a/artifacts

Types

type ArtifactObject 

type ArtifactObject struct {
	Path  string
	Size  int64 // artifact object size in bytes.
	IsDir bool
}

ArtifactObject represents Artifact object agnostic to selected storage.

func (ArtifactObject) GetPath 

func (o ArtifactObject) GetPath() string

GetPath returns Artifact Path.

func (ArtifactObject) GetSize 

func (o ArtifactObject) GetSize() int64

GetSize returns Artifact Size in bytes.

func (ArtifactObject) IsDirectory 

func (o ArtifactObject) IsDirectory() bool

IsDirectory show that object is directly or not.

type ArtifactStorageFactory added in v0.3.2 

type ArtifactStorageFactory struct {
	// contains filtered or unexported fields
}

ArtifactStorageFactory represents Artifact Storage .

func NewArtifactStorageFactory added in v0.3.2 

func NewArtifactStorageFactory(config *config.ServiceConfig) (*ArtifactStorageFactory, error)

NewArtifactStorageFactory creates new Artifact Storage Factory instance.

func (*ArtifactStorageFactory) GetStorage added in v0.3.2 

func (s *ArtifactStorageFactory) GetStorage(
	ctx context.Context,
	runArtifactPath string,
) (ArtifactStorageProvider, error)

GetStorage returns Artifact storage based on provided runArtifactPath.

type ArtifactStorageFactoryProvider added in v0.3.2 

type ArtifactStorageFactoryProvider interface {
	// GetStorage returns Artifact storage based on provided runArtifactPath.
	GetStorage(ctx context.Context, runArtifactPath string) (ArtifactStorageProvider, error)
}

ArtifactStorageFactoryProvider provides an interface provider to work with Artifact Storage.

type ArtifactStorageProvider added in v0.3.2 

type ArtifactStorageProvider interface {
	// Get returns an io.ReadCloser for specific artifact.
	Get(ctx context.Context, artifactURI, path string) (io.ReadCloser, error)
	// List lists all artifact object under provided path.
	List(ctx context.Context, artifactURI, path string) ([]ArtifactObject, error)
}

ArtifactStorageProvider provides an interface to work with artifact storage.

type GS added in v0.4.0 

type GS struct {
	// contains filtered or unexported fields
}

GS represents adapter to work with GS storage artifacts.

func NewGS added in v0.4.0 

func NewGS(ctx context.Context, config *config.ServiceConfig) (*GS, error)

NewGS creates new Google Storage instance.

func (GS) Get added in v0.4.0 

func (s GS) Get(ctx context.Context, artifactURI, path string) (io.ReadCloser, error)

Get returns file content at the storage location.

func (GS) List added in v0.4.0 

func (s GS) List(ctx context.Context, artifactURI, path string) ([]ArtifactObject, error)

List implements ArtifactStorageProvider interface.

type Local 

type Local struct{}

Local represents local file storage adapter to work with artifacts.

func NewLocal 

func NewLocal(config *config.ServiceConfig) (*Local, error)

NewLocal creates new Local storage instance.

func (Local) Get added in v0.3.2 

func (s Local) Get(ctx context.Context, artifactURI, path string) (io.ReadCloser, error)

Get returns actual file content at the storage location.

func (Local) List 

func (s Local) List(ctx context.Context, artifactURI, path string) ([]ArtifactObject, error)

List implements ArtifactStorageProvider interface.

type MockArtifactStorageFactoryProvider added in v0.3.2 

type MockArtifactStorageFactoryProvider struct {
	mock.Mock
}

MockArtifactStorageFactoryProvider is an autogenerated mock type for the ArtifactStorageFactoryProvider type

func NewMockArtifactStorageFactoryProvider added in v0.3.2 

func NewMockArtifactStorageFactoryProvider(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockArtifactStorageFactoryProvider

NewMockArtifactStorageFactoryProvider creates a new instance of MockArtifactStorageFactoryProvider. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockArtifactStorageFactoryProvider) GetStorage added in v0.3.2 

func (_m *MockArtifactStorageFactoryProvider) GetStorage(ctx context.Context, runArtifactPath string) (ArtifactStorageProvider, error)

GetStorage provides a mock function with given fields: ctx, runArtifactPath

type MockArtifactStorageProvider added in v0.3.2 

type MockArtifactStorageProvider struct {
	mock.Mock
}

MockArtifactStorageProvider is an autogenerated mock type for the ArtifactStorageProvider type

func NewMockArtifactStorageProvider added in v0.3.2 

func NewMockArtifactStorageProvider(t interface {
	mock.TestingT
	Cleanup(func())
}) *MockArtifactStorageProvider

NewMockArtifactStorageProvider creates a new instance of MockArtifactStorageProvider. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. The first argument is typically a *testing.T value.

func (*MockArtifactStorageProvider) Get added in v0.3.2 

func (_m *MockArtifactStorageProvider) Get(ctx context.Context, artifactURI string, path string) (io.ReadCloser, error)

Get provides a mock function with given fields: ctx, artifactURI, path

func (*MockArtifactStorageProvider) List added in v0.3.2 

func (_m *MockArtifactStorageProvider) List(ctx context.Context, artifactURI string, path string) ([]ArtifactObject, error)

List provides a mock function with given fields: ctx, artifactURI, path

type S3 

type S3 struct {
	// contains filtered or unexported fields
}

S3 represents S3 adapter to work with artifacts.

func NewS3 

func NewS3(ctx context.Context, config *config.ServiceConfig) (*S3, error)

NewS3 creates new S3 instance.

func (S3) Get added in v0.3.2 

func (s S3) Get(ctx context.Context, artifactURI, path string) (io.ReadCloser, error)

Get returns file content at the storage location.

func (S3) List 

func (s S3) List(ctx context.Context, artifactURI, path string) ([]ArtifactObject, error)

List implements ArtifactStorageProvider interface.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.