persistence

package
v0.0.0-...-b29c596 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 28, 2023 License: Apache-2.0 Imports: 15 Imported by: 0

README

Storage Location

The default storage driver url is file:///tmp/mediorum/blobs. This is the directory on your node where files will stored post transcoding.

To update the local storage location set the AUDIUS_STORAGE_DRIVER_URL environment variable

AUDIUS_STORAGE_DRIVER_URL="file:///my/directory"

The driver is determined based on the storage_url (eg s3://your-bucket uses the s3 driver)

💡 Note: URLs beginning with https:// will default to s3

Cloud Storage

To use cloud object storage instead of local storage, you will need to:

  • create either an s3 bucket, gcs bucket or azure storage container
  • your bucket access policy should NOT BE PUBLIC
  • provision credentials that allow object storage access

S3 storage

AWS_ACCESS_KEY_ID="<my-access-key-id>"
AWS_SECRET_ACCESS_KEY="<my-secret-access-key>"
AWS_REGION="<my-aws-region>"
AUDIUS_STORAGE_DRIVER_URL="s3://my-bucket?region=us-west-1"

GCS storage

GOOGLE_APPLICATION_CREDENTIALS="/path/to/creds.json"
AUDIUS_STORAGE_DRIVER_URL="gs://my-bucket"

Azure storage

AZURE_STORAGE_ACCOUNT="<storage-account-name>"
AZURE_STORAGE_KEY="<storage-key>"
AUDIUS_STORAGE_DRIVER_URL="azblob://my-container"

Documentation

Overview

Package persistence handles taking content from the temp store and storing it in the persistence store.

Index

Constants

View Source
const (
	AWS_ACCESS_KEY_ID     = "AWS_ACCESS_KEY_ID"
	AWS_SECRET_ACCESS_KEY = "AWS_SECRET_ACCESS_KEY"
	AWS_REGION            = "AWS_REGION"

	GOOGLE_APPLICATION_CREDENTIALS = "GOOGLE_APPLICATION_CREDENTIALS"

	AZURE_STORAGE_ACCOUNT = "AZURE_STORAGE_ACCOUNT"
	AZURE_STORAGE_KEY     = "AZURE_STORAGE_KEY"
)

Variables

This section is empty.

Functions

func MoveAllFiles

func MoveAllFiles(from, to *blob.Bucket) error

func Open

func Open(blobDriverURL string) (*blob.Bucket, error)

New creates a struct that listens to streamToStoreFrom and downloads content from the temp store to the persistent store.

Types

type Prefix

type Prefix int
const (
	FILE Prefix = iota
	HTTP
	GS
	S3
	AZBLOB
)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL