arangodb

package
v0.0.0-...-3682eb9 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 6, 2024 License: Apache-2.0 Imports: 17 Imported by: 0

README

##arangodb

database: https://github.com/arangodb/arangodb go driver: https://github.com/arangodb/go-driver

options:

[arangodb]
enabled=true
db_name="seaweedfs"
servers=["http://localhost:8529"]
#basic auth
user="root"
pass="test"

# tls settings
insecure_skip_verify=true

i test using this dev database: docker run -p 8529:8529 -e ARANGO_ROOT_PASSWORD=test arangodb/arangodb:3.9.0

database structure

arangodb has a few restrictions which require the use of a few tricks in order to losslessly store the data.

filer store

arangodb does not support []byte, and will store such as a uint64 array. this would be a waste of space. to counteract this, we store the data as a length prefixed uint64 byteset.

filer kv

same as above

filer buckets

s3 buckets are implemented through arangodb collection. this allows us to do very fast bucket deletion by simply deleting the collection

arangodb collection name rules is character set azAZ09_- with a 256 character max. however the first character must be a letter.

s3 bucket name rule is the set azAZ09.- with a 63 characters max.

the rules for collection names is then the following:

  1. if the bucket name is a valid arangodb collection name, then nothing is done.
  2. if the bucket name contains a ".", the "." is replaced with "_"
  3. if the bucket name now begins with a number or "_", the prefix "xN--" is prepended to the collection name

this allows for these collection names to be used.

features i don't personally need but are missing

[ ] provide tls cert to arango [ ] authentication that is not basic auth [ ] synchronise endpoint interval config [ ] automatic creation of custom index [ ] configure default arangodb collection sharding rules [ ] configure default arangodb collection replication rules

complexity

ok, so if https://www.arangodb.com/docs/stable/indexing-index-basics.html#persistent-index is correct

O(1)

  • InsertEntry
  • UpdateEntry
  • FindEntry
  • DeleteEntry
  • KvPut
  • KvGet
  • KvDelete

O(log(BUCKET_SIZE))

  • DeleteFolderChildren

O(log(DIRECTORY_SIZE))

  • ListDirectoryEntries
  • ListDirectoryPrefixedEntries

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	BUCKET_PREFIX      = "/buckets"
	DEFAULT_COLLECTION = "seaweed_no_bucket"
	KVMETA_COLLECTION  = "seaweed_kvmeta"
)

Functions

This section is empty.

Types

type ArangodbStore

type ArangodbStore struct {
	// contains filtered or unexported fields
}

func (*ArangodbStore) BeginTransaction

func (store *ArangodbStore) BeginTransaction(ctx context.Context) (context.Context, error)

func (*ArangodbStore) CanDropWholeBucket

func (store *ArangodbStore) CanDropWholeBucket() bool

func (*ArangodbStore) CommitTransaction

func (store *ArangodbStore) CommitTransaction(ctx context.Context) error

func (*ArangodbStore) DeleteEntry

func (store *ArangodbStore) DeleteEntry(ctx context.Context, fullpath util.FullPath) (err error)

func (*ArangodbStore) DeleteFolderChildren

func (store *ArangodbStore) DeleteFolderChildren(ctx context.Context, fullpath util.FullPath) (err error)

this runs in log time

func (*ArangodbStore) FindEntry

func (store *ArangodbStore) FindEntry(ctx context.Context, fullpath util.FullPath) (entry *filer.Entry, err error)

func (*ArangodbStore) GetName

func (store *ArangodbStore) GetName() string

func (*ArangodbStore) Initialize

func (store *ArangodbStore) Initialize(configuration util.Configuration, prefix string) (err error)

func (*ArangodbStore) InsertEntry

func (store *ArangodbStore) InsertEntry(ctx context.Context, entry *filer.Entry) (err error)

func (*ArangodbStore) KvDelete

func (store *ArangodbStore) KvDelete(ctx context.Context, key []byte) (err error)

func (*ArangodbStore) KvGet

func (store *ArangodbStore) KvGet(ctx context.Context, key []byte) (value []byte, err error)

func (*ArangodbStore) KvPut

func (store *ArangodbStore) KvPut(ctx context.Context, key []byte, value []byte) (err error)

func (*ArangodbStore) ListDirectoryEntries

func (store *ArangodbStore) ListDirectoryEntries(ctx context.Context, dirPath util.FullPath, startFileName string, includeStartFile bool, limit int64, eachEntryFunc filer.ListEachEntryFunc) (lastFileName string, err error)

func (*ArangodbStore) ListDirectoryPrefixedEntries

func (store *ArangodbStore) ListDirectoryPrefixedEntries(ctx context.Context, dirPath util.FullPath, startFileName string, includeStartFile bool, limit int64, prefix string, eachEntryFunc filer.ListEachEntryFunc) (lastFileName string, err error)

func (*ArangodbStore) OnBucketCreation

func (store *ArangodbStore) OnBucketCreation(bucket string)

func (*ArangodbStore) OnBucketDeletion

func (store *ArangodbStore) OnBucketDeletion(bucket string)

func (*ArangodbStore) RollbackTransaction

func (store *ArangodbStore) RollbackTransaction(ctx context.Context) error

func (*ArangodbStore) Shutdown

func (store *ArangodbStore) Shutdown()

func (*ArangodbStore) UpdateEntry

func (store *ArangodbStore) UpdateEntry(ctx context.Context, entry *filer.Entry) (err error)

type Model

type Model struct {
	Key       string `json:"_key"`
	Directory string `json:"directory,omitempty"`
	Name      string `json:"name,omitempty"`
	Ttl       string `json:"ttl,omitempty"`

	//arangodb does not support binary blobs
	//we encode byte slice into uint64 slice
	//see helpers.go
	Meta []uint64 `json:"meta"`
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL