cmn

package
v1.3.22 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 25, 2024 License: MIT Imports: 38 Imported by: 6

Documentation

Overview

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2023, NVIDIA CORPORATION. All rights reserved.

msgp -file cmn/objlist.go -tests=false -marshal=false -unexported Code generated by the command above where msgp is tinylib/msgp; see docs/msgp.md. DO NOT EDIT.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2023, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2018-2024, NVIDIA CORPORATION. All rights reserved.

Package cmn provides common constants, types, and utilities for AIS clients and AIStore.

  • Copyright (c) 2022-2024, NVIDIA CORPORATION. All rights reserved.

Index

Constants

View Source
const (
	PropBucketAccessAttrs  = "access"             // Bucket access attributes.
	PropBucketVerEnabled   = "versioning.enabled" // Enable/disable object versioning in a bucket.
	PropBucketCreated      = "created"            // Bucket creation time.
	PropBackendBck         = "backend_bck"
	PropBackendBckName     = PropBackendBck + ".name"
	PropBackendBckProvider = PropBackendBck + ".provider"
)
View Source
const (
	IgnoreReaction = "ignore"
	WarnReaction   = "warn"
	AbortReaction  = "abort"
)

dsort

View Source
const (
	FmtErrIntegrity      = "[%s%d, for troubleshooting see %s/blob/main/docs/troubleshooting.md]"
	FmtErrUnmarshal      = "%s: failed to unmarshal %s (%s), err: %w"
	FmtErrMorphUnmarshal = "%s: failed to unmarshal %s (%T), err: %w"
	FmtErrUnknown        = "%s: unknown %s %q"
	FmtErrBackwardCompat = "%v (backward compatibility is supported only one version back, e.g. 3.9 => 3.10)"

	BadSmapPrefix = "[bad cluster map]"

	StartupMayTimeout = "cluster startup is taking unusually long time..." // related ErrStartupTimeout
)
View Source
const (
	RetryLogVerbose = iota
	RetryLogQuiet
	RetryLogOff
)
View Source
const (
	NetPublic       = "PUBLIC"
	NetIntraControl = "INTRA-CONTROL"
	NetIntraData    = "INTRA-DATA"
)
View Source
const (
	DefaultMaxIdleConns        = 64
	DefaultMaxIdleConnsPerHost = 16
	DefaultIdleConnTimeout     = 8 * time.Second
	DefaultWriteBufferSize     = 64 * cos.KiB
	DefaultReadBufferSize      = 64 * cos.KiB
	DefaultSendRecvBufferSize  = 128 * cos.KiB
)

http.DefaultTransport has the following defaults: - MaxIdleConns: 100, - MaxIdleConnsPerHost : 2 (via DefaultMaxIdleConnsPerHost) - IdleConnTimeout: 90 * time.Second, - WriteBufferSize: 4KB - ReadBufferSize: 4KB Following are the constants we use by default:

View Source
const (
	// source of the cold-GET and download; the values include all
	// 3rd party backend providers
	SourceObjMD = "source"

	// downloader' source is "web"
	WebObjMD = "web"

	VersionObjMD = "version" // "generation" for GCP, "version" for AWS but only if the bucket is versioned, etc.
	CRC32CObjMD  = cos.ChecksumCRC32C
	MD5ObjMD     = cos.ChecksumMD5
	ETag         = cos.HdrETag

	OrigURLObjMD = "orig_url"

	// additional backend
	LastModified = "LastModified"
)

LOM custom metadata stored under `lomCustomMD`.

View Source
const (
	VersionAIStore = "3.22"
	VersionCLI     = "1.10"
	VersionLoader  = "1.10"
	VersionAuthN   = "1.0"
)
View Source
const (
	MetaverSmap  = 2 // Smap (cluster map) formatting version a.k.a. meta-version (see core/meta/jsp.go)
	MetaverBMD   = 2 // BMD (bucket metadata) --/--
	MetaverRMD   = 1 // Rebalance MD (jsp)
	MetaverVMD   = 1 // Volume MD (jsp)
	MetaverEtlMD = 1 // ETL MD (jsp)

	MetaverLOM = 1 // LOM

	MetaverConfig      = 3 // Global Configuration (jsp)
	MetaverAuthNConfig = 1 // Authn config (jsp) // ditto
	MetaverAuthTokens  = 1 // Authn tokens (jsp) // ditto

	MetaverMetasync = 1 // metasync over network formatting version (jsp)

	MetaverJSP = jsp.Metaver // `jsp` own encoding version
)
View Source
const AwsDefaultRegion = "us-east-1"
View Source
const AwsMultipartDelim = "-"

from https://docs.aws.amazon.com/AmazonS3/latest/API/API_Object.html "The ETag may or may not be an MD5 digest of the object data. Whether or not it is depends on how the object was created and how it is encrypted..."

View Source
const GitHubHome = "https://github.com/NVIDIA/aistore"
View Source
const HostnameListSepa = ","
View Source
const IterFieldNameSepa = "."
View Source
const MsgpLsoBufSize = 32 * cos.KiB
View Source
const (
	// NsGlobalUname is hardcoded here to avoid allocating it via Uname()
	// (the most common use case)
	NsGlobalUname = "@#"
)
View Source
const (
	ObjSizeToAlwaysReplicate = -1 // (see `ObjSizeLimit` comment above)

)

Variables

View Source
var (
	// NsGlobal represents *this* cluster's global namespace that is used by default when
	// no specific namespace was defined or provided by the user.
	NsGlobal = Ns{}
	// NsAnyRemote represents any remote cluster. As such, NsGlobalRemote applies
	// exclusively to AIS (provider) given that other Backend providers are remote by definition.
	NsAnyRemote = Ns{UUID: string(apc.NsUUIDPrefix)}
)
View Source
var (
	ErrSkip             = errors.New("skip")
	ErrStartupTimeout   = errors.New("startup timeout") // related StartupMayTimeout
	ErrQuiesceTimeout   = errors.New("timed out waiting for quiescence")
	ErrNotEnoughTargets = errors.New("not enough target nodes")
	ErrNoMountpaths     = errors.New("no mountpaths")

	// aborts
	ErrXactRenewAbort   = errors.New("renewal abort")
	ErrXactUserAbort    = errors.New("user abort")              // via apc.ActXactStop
	ErrXactICNotifAbort = errors.New("IC(notifications) abort") // ditto
)
View Source
var BackendHelpers = struct {
	Amazon backendFuncs
	Azure  backendFuncs
	Google backendFuncs
	HDFS   backendFuncs
	HTTP   backendFuncs
}{
	Amazon: backendFuncs{
		EncodeVersion: func(v any) (string, bool) {
			switch x := v.(type) {
			case *string:
				if awsIsVersionSet(x) {
					return *x, true
				}
				return "", false
			case string:
				if awsIsVersionSet(&x) {
					return x, true
				}
				return x, false
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
		EncodeCksum: func(v any) (string, bool) {
			switch x := v.(type) {
			case *string:
				if IsS3MultipartEtag(*x) {
					return *x, true
				}
				return UnquoteCEV(*x), true
			case string:
				return x, true
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
	},
	Google: backendFuncs{
		EncodeVersion: func(v any) (string, bool) {
			switch x := v.(type) {
			case string:
				return x, x != ""
			case int64:
				return strconv.FormatInt(x, 10), true
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
		EncodeCksum: func(v any) (string, bool) {
			switch x := v.(type) {
			case string:
				decoded, err := base64.StdEncoding.DecodeString(x)
				if err != nil {
					return "", false
				}
				return hex.EncodeToString(decoded), true
			case []byte:
				return hex.EncodeToString(x), true
			case uint32:

				b := []byte{byte(x >> 24), byte(x >> 16), byte(x >> 8), byte(x)}
				return base64.StdEncoding.EncodeToString(b), true
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
	},
	HDFS: backendFuncs{
		EncodeCksum: func(v any) (cksumValue string, isSet bool) {
			switch x := v.(type) {
			case []byte:
				return hex.EncodeToString(x), true
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
	},
	HTTP: backendFuncs{
		EncodeVersion: func(v any) (string, bool) {
			switch x := v.(type) {
			case string:

				x = strings.TrimPrefix(x, "W/")
				x = UnquoteCEV(x)
				return x, x != ""
			default:
				debug.FailTypeCast(v)
				return "", false
			}
		},
	},
}
View Source
var ConfigRestartRequired = []string{"auth", "memsys", "net"}

assorted named fields that require (cluster | node) restart for changes to make an effect

View Source
var GCO *gco
View Source
var Rom readMostly

Functions

func CustomMD2S

func CustomMD2S(md cos.StrKVs) string

func DelBckFromQuery

func DelBckFromQuery(query url.Values) url.Values

func DirHasOrIsPrefix added in v1.3.16

func DirHasOrIsPrefix(dirPath, prefix string) bool

Directory has to either: - include (or match) prefix, or - be contained in prefix - motivation: don't SkipDir a/b when looking for a/b/c An alternative name for this function could be smth. like SameBranch()

func EnvToTLS added in v1.3.21

func EnvToTLS(sargs *TLSArgs)

see related: HTTPConf.ToTLS()

func FreeBuffer added in v1.3.18

func FreeBuffer(buf *bytes.Buffer)

func FreeHra

func FreeHra(a *HreqArgs)

func FreeHterr

func FreeHterr(a *ErrHTTP)

func InitErrs added in v1.3.18

func InitErrs(a string, b func(error))

func IsErrAborted

func IsErrAborted(err error) bool

func IsErrBckNotFound

func IsErrBckNotFound(err error) bool

func IsErrBucketAlreadyExists

func IsErrBucketAlreadyExists(err error) bool

func IsErrBucketLevel

func IsErrBucketLevel(err error) bool

func IsErrBucketNought

func IsErrBucketNought(err error) bool

nought: not a thing

func IsErrCapExceeded added in v1.3.19

func IsErrCapExceeded(err error) bool

func IsErrLmetaCorrupted

func IsErrLmetaCorrupted(err error) bool

func IsErrLmetaNotFound

func IsErrLmetaNotFound(err error) bool

func IsErrMountpathNotFound

func IsErrMountpathNotFound(err error) bool

func IsErrObjLevel

func IsErrObjLevel(err error) bool

func IsErrObjNought

func IsErrObjNought(err error) bool

lom.Load

func IsErrRangeNotSatisfiable added in v1.3.22

func IsErrRangeNotSatisfiable(err error) bool

func IsErrRemoteBckNotFound

func IsErrRemoteBckNotFound(err error) bool

func IsErrSoft

func IsErrSoft(err error) bool

func IsErrStreamTerminated added in v1.3.16

func IsErrStreamTerminated(err error) bool

func IsErrXactNotFound added in v1.3.16

func IsErrXactNotFound(err error) bool

func IsErrXactUsePrev added in v1.3.18

func IsErrXactUsePrev(err error) bool

func IsFileAlreadyClosed added in v1.3.16

func IsFileAlreadyClosed(err error) bool

func IsNestedMpath

func IsNestedMpath(a string, la int, b string) (err error)

func IsS3MultipartEtag added in v1.3.22

func IsS3MultipartEtag(etag string) bool

func IsStatusBadGateway

func IsStatusBadGateway(err error) (yes bool)

func IsStatusGone

func IsStatusGone(err error) (yes bool)

func IsStatusNotFound

func IsStatusNotFound(err error) (yes bool)

func IsStatusServiceUnavailable

func IsStatusServiceUnavailable(err error) (yes bool)

func IterFields

func IterFields(v any, updf updateFunc, opts ...IterOpts) error

IterFields walks the struct and calls `updf` callback at every leaf field that it encounters. The (nested) names are created by joining the json tag with dot. Iteration supports reading another, custom tag `list` with values:

  • `tagOmitempty` - omit empty fields (only for read run)
  • `tagOmit` - omit field
  • `tagReadonly` - field cannot be updated (returns error on `SetValue`)

Examples of usages for tags can be found in `BucketProps` or `Config` structs.

Passing additional options with `IterOpts` can for example call callback also at the non-leaf structures.

func KeepaliveRetryDuration

func KeepaliveRetryDuration(c *Config) time.Duration

func LoadConfig

func LoadConfig(globalConfPath, localConfPath, daeRole string, config *Config) error

is called at startup

func MakeRangeHdr added in v1.3.16

func MakeRangeHdr(start, length int64) string

Ref: https://www.rfc-editor.org/rfc/rfc7233#section-2.1 (compare w/ htrange.contentRange)

func MaxParallelism added in v1.3.22

func MaxParallelism() int

func NetworkCallWithRetry

func NetworkCallWithRetry(args *RetryArgs) (err error)

func NetworkIsKnown

func NetworkIsKnown(net string) bool

func NewBuffer added in v1.3.18

func NewBuffer() (buf *bytes.Buffer)

func NewClient

func NewClient(cargs TransportArgs) *http.Client

NOTE: `NewTransport` (below) fills-in certain defaults

func NewClientTLS added in v1.3.21

func NewClientTLS(cargs TransportArgs, sargs TLSArgs) *http.Client

https client (ditto)

func NewDefaultClients added in v1.3.22

func NewDefaultClients(timeout time.Duration) (clientH, clientTLS *http.Client)

func NewIntraClientTLS added in v1.3.21

func NewIntraClientTLS(cargs TransportArgs, config *Config) *http.Client

func NewTLS added in v1.3.21

func NewTLS(sargs TLSArgs) (tlsConf *tls.Config, _ error)

func NewTransport

func NewTransport(cargs TransportArgs) *http.Transport

{TransportArgs + defaults} => http.Transport for a variety of ais clients NOTE: TLS below, and separately

func NormalizeProvider

func NormalizeProvider(provider string) (p string, err error)

func ObjHasPrefix added in v1.3.16

func ObjHasPrefix(objName, prefix string) bool

func OrigURLBck2Name

func OrigURLBck2Name(origURLBck string) (bckName string)

func ParsePort

func ParsePort(p string) (int, error)

func ParseReadHeaderTimeout added in v1.3.22

func ParseReadHeaderTimeout() (_ time.Duration, isSet bool)

func ParseURL added in v1.3.18

func ParseURL(path string, itemsPresent []string, itemsAfter int, splitAfter bool) ([]string, error)

ParseURL splits URL path at "/" and matches resulting items against the specified, if any. - splitAfter == true: strings.Split() the entire path; - splitAfter == false: strings.SplitN(len(itemsPresent)+itemsAfter) Returns all items that follow the specified `items`.

func ParseURLScheme

func ParseURLScheme(url string) (scheme, address string)

Splits url into [(scheme)://](address). It's not possible to use url.Parse as (from url.Parse() docs) 'Trying to parse a hostname and path without a scheme is invalid'

func PrependProtocol

func PrependProtocol(url string, protocol ...string) string

PrependProtocol prepends protocol in URL in case it is missing. By default it adds `http://` to the URL.

func ReadBytes

func ReadBytes(r *http.Request) (b []byte, err error)

func ReadJSON

func ReadJSON(w http.ResponseWriter, r *http.Request, out any) (err error)

func S2CustomMD added in v1.3.22

func S2CustomMD(custom, version string) (md cos.StrKVs)

func SaveOverrideConfig

func SaveOverrideConfig(configDir string, toUpdate *ConfigToSet) error

func SortLso

func SortLso(entries LsoEntries)

func ToHeader

func ToHeader(oah cos.OAH, hdr http.Header)

func TokenGreaterEQ

func TokenGreaterEQ(token, objName string) bool

Returns true if the continuation token >= object's name (in other words, the object is already listed and must be skipped). Note that string `>=` is lexicographic.

func TypeCodeHTTPErr added in v1.3.19

func TypeCodeHTTPErr(s string) (tcode string)

func UnquoteCEV added in v1.3.22

func UnquoteCEV(val string) string

unquote checksum, ETag, and version e.g., https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag

func UpdateFieldValue

func UpdateFieldValue(s any, name string, value any) error

UpdateFieldValue updates the field in the struct with given value. Returns error if the field was not found or could not be updated.

func ValidateMpath

func ValidateMpath(mpath string) (string, error)

common mountpath validation (NOTE: calls filepath.Clean() every time)

func ValidatePort

func ValidatePort(port int) (int, error)

func ValidatePrefix added in v1.3.18

func ValidatePrefix(s string) error

ditto

func ValidateRemAlias

func ValidateRemAlias(alias string) (err error)

func WriteErr

func WriteErr(w http.ResponseWriter, r *http.Request, err error, opts ...int)

sends HTTP response header with the provided status (alloc/free via mem-pool)

func WriteErr405

func WriteErr405(w http.ResponseWriter, r *http.Request, methods ...string)

405 Method Not Allowed, see: * https://www.rfc-editor.org/rfc/rfc7231#section-6.5.5

func WriteErrJSON

func WriteErrJSON(w http.ResponseWriter, r *http.Request, out any, err error) error

func WriteErrMsg

func WriteErrMsg(w http.ResponseWriter, r *http.Request, msg string, opts ...int)

Create ErrHTTP (based on `msg` and `opts`) and write it into HTTP response.

Types

type AllBsummResults

type AllBsummResults []*BsummResult

func (AllBsummResults) Aggregate

func (s AllBsummResults) Aggregate(from *BsummResult) AllBsummResults

func (AllBsummResults) Finalize

func (s AllBsummResults) Finalize(dsize map[string]uint64, testingEnv bool)

func (AllBsummResults) Len

func (s AllBsummResults) Len() int

func (AllBsummResults) Less

func (s AllBsummResults) Less(i, j int) bool

func (AllBsummResults) Swap

func (s AllBsummResults) Swap(i, j int)

type ArchiveBckMsg added in v1.3.18

type ArchiveBckMsg struct {
	ToBck Bck `json:"tobck"`
	apc.ArchiveMsg
}

ArchiveBckMsg contains parameters to archive mutiple objects from the specified (source) bucket. Destination bucket may the same as the source or a different one. -------------------- NOTE on terminology: --------------------- "archive" is any (.tar, .tgz/.tar.gz, .zip, .tar.lz4) formatted object often also called "shard"

See also: apc.PutApndArchArgs

func (*ArchiveBckMsg) Cname added in v1.3.18

func (msg *ArchiveBckMsg) Cname() string

type AuthConf

type AuthConf struct {
	Secret  string `json:"secret"`
	Enabled bool   `json:"enabled"`
}

global configuration

type AuthConfToSet added in v1.3.21

type AuthConfToSet struct {
	Secret  *string `json:"secret,omitempty"`
	Enabled *bool   `json:"enabled,omitempty"`
}

global configuration

type BackendBckToSet added in v1.3.21

type BackendBckToSet struct {
	Name     *string `json:"name"`
	Provider *string `json:"provider"`
}

type BackendConf

type BackendConf struct {
	// provider implementation-dependent
	Conf map[string]any `json:"conf,omitempty"`
	// 3rd party Cloud(s) -- set during validation
	Providers map[string]Ns `json:"-"`
}

global configuration

func (*BackendConf) EqualClouds

func (c *BackendConf) EqualClouds(o *BackendConf) bool

func (*BackendConf) EqualRemAIS

func (c *BackendConf) EqualRemAIS(o *BackendConf, sname string) bool

func (*BackendConf) Get added in v1.3.16

func (c *BackendConf) Get(provider string) (conf any)

func (*BackendConf) MarshalJSON

func (c *BackendConf) MarshalJSON() (data []byte, err error)

func (*BackendConf) Set added in v1.3.16

func (c *BackendConf) Set(provider string, newConf any)

func (*BackendConf) UnmarshalJSON

func (c *BackendConf) UnmarshalJSON(data []byte) error

func (*BackendConf) Validate

func (c *BackendConf) Validate() (err error)

type BackendConfAIS

type BackendConfAIS map[string][]string // cluster alias -> [urls...]

global configuration

func (BackendConfAIS) String

func (c BackendConfAIS) String() (s string)

type BackendConfHDFS

type BackendConfHDFS struct {
	Addresses           []string `json:"addresses"`
	User                string   `json:"user"`
	UseDatanodeHostname bool     `json:"use_datanode_hostname"`
}

global configuration

type Bck

type Bck struct {
	Props    *Bprops `json:"-"`
	Name     string  `json:"name" yaml:"name"`
	Provider string  `json:"provider" yaml:"provider"` // NOTE: see api/apc/provider.go for supported enum
	Ns       Ns      `json:"namespace" yaml:"namespace" list:"omitempty"`
}

func ParseBckObjectURI

func ParseBckObjectURI(uri string, opts ParseURIOpts) (bck Bck, objName string, err error)

func ParseUname

func ParseUname(uname string) (b Bck, objName string)

unique name => Bck (use MakeUname above to perform the reverse translation)

func (*Bck) AddToQuery

func (b *Bck) AddToQuery(query url.Values) url.Values

func (*Bck) AddUnameToQuery

func (b *Bck) AddUnameToQuery(query url.Values, uparam string) url.Values

func (*Bck) Backend

func (b *Bck) Backend() *Bck

func (*Bck) Cname added in v1.3.16

func (b *Bck) Cname(objname string) (s string)

canonical name, with or without object

func (*Bck) Copy

func (b *Bck) Copy(src *Bck)

func (*Bck) DefaultProps

func (bck *Bck) DefaultProps(c *ClusterConfig) *Bprops

By default, created buckets inherit their properties from the cluster (global) configuration. Global configuration, in turn, is protected versioned, checksummed, and replicated across the entire cluster.

* Bucket properties can be changed at any time via `api.SetBprops`. * In addition, `api.CreateBucket` allows to specify (non-default) properties at bucket creation time. * Inherited defaults include checksum, LRU, etc. configurations - see below. * By default, LRU is disabled for AIS (`ais://`) buckets.

See also:

  • github.com/NVIDIA/aistore/blob/main/docs/bucket.md#default-bucket-properties
  • BpropsToSet (above)
  • ais.defaultBckProps()

func (Bck) Equal

func (b Bck) Equal(other *Bck) bool

func (*Bck) HasProvider

func (b *Bck) HasProvider() bool

func (*Bck) HasVersioningMD added in v1.3.22

func (b *Bck) HasVersioningMD() bool

A subset of remote backends that maintain assorted items of versioning information - the items including ETag, checksum, etc. - that, in turn, can be used to populate `ObjAttrs` * see related: `ObjAttrs.Equal`

func (*Bck) IsAIS

func (b *Bck) IsAIS() bool

func (*Bck) IsCloud

func (b *Bck) IsCloud() bool

func (*Bck) IsEmpty

func (b *Bck) IsEmpty() bool

func (*Bck) IsHDFS

func (b *Bck) IsHDFS() bool

func (*Bck) IsHTTP

func (b *Bck) IsHTTP() bool

func (*Bck) IsQuery

func (b *Bck) IsQuery() bool

QueryBcks (see below) is a Bck that _can_ have an empty Name.

func (*Bck) IsRemote

func (b *Bck) IsRemote() bool

func (*Bck) IsRemoteAIS

func (b *Bck) IsRemoteAIS() bool

func (*Bck) LenUnameGlob added in v1.3.21

func (b *Bck) LenUnameGlob(objName string) int

func (*Bck) Less

func (b *Bck) Less(other *Bck) bool

func (*Bck) MakeUname

func (b *Bck) MakeUname(objName string) string

Bck => unique name (use ParseUname below to translate back)

func (*Bck) NewQuery added in v1.3.21

func (b *Bck) NewQuery() (q url.Values)

func (*Bck) RemoteBck

func (b *Bck) RemoteBck() *Bck

func (Bck) String

func (b Bck) String() (s string)

func (*Bck) Validate

func (b *Bck) Validate() (err error)

func (*Bck) ValidateName

func (b *Bck) ValidateName() (err error)

type Bcks

type Bcks []Bck

func (Bcks) Equal

func (bcks Bcks) Equal(other Bcks) bool

func (Bcks) Len

func (bcks Bcks) Len() int

func (Bcks) Less

func (bcks Bcks) Less(i, j int) bool

func (Bcks) Select

func (bcks Bcks) Select(query QueryBcks) (filtered Bcks)

func (Bcks) Swap

func (bcks Bcks) Swap(i, j int)

type Bprops added in v1.3.21

type Bprops struct {
	BackendBck  Bck             `json:"backend_bck,omitempty"` // makes remote bucket out of a given ais bucket
	Extra       ExtraProps      `json:"extra,omitempty" list:"omitempty"`
	WritePolicy WritePolicyConf `json:"write_policy"`
	Provider    string          `json:"provider" list:"readonly"`       // backend provider
	Renamed     string          `list:"omit"`                           // non-empty if the bucket has been renamed
	Cksum       CksumConf       `json:"checksum"`                       // the bucket's checksum
	EC          ECConf          `json:"ec"`                             // erasure coding
	LRU         LRUConf         `json:"lru"`                            // LRU (watermarks and enabled/disabled)
	Mirror      MirrorConf      `json:"mirror"`                         // mirroring
	Access      apc.AccessAttrs `json:"access,string"`                  // access permissions
	BID         uint64          `json:"bid,string" list:"omit"`         // unique ID
	Created     int64           `json:"created,string" list:"readonly"` // creation timestamp
	Versioning  VersionConf     `json:"versioning"`                     // versioning (see "inherit")
}

func (*Bprops) Apply added in v1.3.21

func (bp *Bprops) Apply(propsToSet *BpropsToSet)

func (*Bprops) Clone added in v1.3.21

func (bp *Bprops) Clone() *Bprops

func (*Bprops) Equal added in v1.3.21

func (bp *Bprops) Equal(other *Bprops) (eq bool)

func (*Bprops) SetProvider added in v1.3.21

func (bp *Bprops) SetProvider(provider string)

func (*Bprops) Validate added in v1.3.21

func (bp *Bprops) Validate(targetCnt int) error

type BpropsToSet added in v1.3.21

type BpropsToSet struct {
	BackendBck  *BackendBckToSet      `json:"backend_bck,omitempty"`
	Versioning  *VersionConfToSet     `json:"versioning,omitempty"`
	Cksum       *CksumConfToSet       `json:"checksum,omitempty"`
	LRU         *LRUConfToSet         `json:"lru,omitempty"`
	Mirror      *MirrorConfToSet      `json:"mirror,omitempty"`
	EC          *ECConfToSet          `json:"ec,omitempty"`
	Access      *apc.AccessAttrs      `json:"access,string,omitempty"`
	WritePolicy *WritePolicyConfToSet `json:"write_policy,omitempty"`
	Extra       *ExtraToSet           `json:"extra,omitempty"`
	Force       bool                  `json:"force,omitempty" copy:"skip" list:"omit"`
}

Once validated, BpropsToSet are copied to Bprops. The struct may have extra fields that do not exist in Bprops. Add tag 'copy:"skip"' to ignore those fields when copying values.

func NewBpropsToSet added in v1.3.21

func NewBpropsToSet(nvs cos.StrKVs) (props *BpropsToSet, err error)

type BsummResult

type BsummResult struct {
	Bck
	apc.BsummResult
}

type CksumConf

type CksumConf struct {
	// (note that `ChecksumNone` ("none") disables checksumming)
	Type string `json:"type"`

	// validate the checksum of the object that we cold-GET
	// or download from remote location (e.g., cloud bucket)
	ValidateColdGet bool `json:"validate_cold_get"`

	// validate object's version (if exists and provided) and its checksum -
	// if either value fail to match, the object is removed from ais.
	//
	// NOTE: object versioning is backend-specific and is may _not_ be supported by a given
	// (supported) backends - see docs for details.
	ValidateWarmGet bool `json:"validate_warm_get"`

	// determines whether to validate checksums of objects
	// migrated or replicated within the cluster
	ValidateObjMove bool `json:"validate_obj_move"`

	// EnableReadRange: Return read range checksum otherwise return entire object checksum.
	EnableReadRange bool `json:"enable_read_range"`
}

global configuration

func (*CksumConf) String

func (c *CksumConf) String() string

func (*CksumConf) Validate

func (c *CksumConf) Validate() (err error)

func (*CksumConf) ValidateAsProps

func (c *CksumConf) ValidateAsProps(...any) (err error)

type CksumConfToSet added in v1.3.21

type CksumConfToSet struct {
	Type            *string `json:"type,omitempty"`
	ValidateColdGet *bool   `json:"validate_cold_get,omitempty"`
	ValidateWarmGet *bool   `json:"validate_warm_get,omitempty"`
	ValidateObjMove *bool   `json:"validate_obj_move,omitempty"`
	EnableReadRange *bool   `json:"enable_read_range,omitempty"`
}

global configuration

type ClientConf

type ClientConf struct {
	Timeout        cos.Duration `json:"client_timeout"`
	TimeoutLong    cos.Duration `json:"client_long_timeout"`
	ListObjTimeout cos.Duration `json:"list_timeout"`
}

global configuration

func (*ClientConf) Validate

func (c *ClientConf) Validate() error

type ClientConfToSet added in v1.3.21

type ClientConfToSet struct {
	Timeout        *cos.Duration `json:"client_timeout,omitempty"` // readonly as far as intra-cluster
	TimeoutLong    *cos.Duration `json:"client_long_timeout,omitempty"`
	ListObjTimeout *cos.Duration `json:"list_timeout,omitempty"`
}

global configuration

type ClusterConfig

type ClusterConfig struct {
	Ext        any            `json:"ext,omitempty"` // within meta-version extensions
	Backend    BackendConf    `json:"backend" allow:"cluster"`
	Mirror     MirrorConf     `json:"mirror" allow:"cluster"`
	EC         ECConf         `json:"ec" allow:"cluster"`
	Log        LogConf        `json:"log"`
	Periodic   PeriodConf     `json:"periodic"`
	Timeout    TimeoutConf    `json:"timeout"`
	Client     ClientConf     `json:"client"`
	Proxy      ProxyConf      `json:"proxy" allow:"cluster"`
	Space      SpaceConf      `json:"space"`
	LRU        LRUConf        `json:"lru"`
	Disk       DiskConf       `json:"disk"`
	Rebalance  RebalanceConf  `json:"rebalance" allow:"cluster"`
	Resilver   ResilverConf   `json:"resilver"`
	Cksum      CksumConf      `json:"checksum"`
	Versioning VersionConf    `json:"versioning" allow:"cluster"`
	Net        NetConf        `json:"net"`
	FSHC       FSHCConf       `json:"fshc"`
	Auth       AuthConf       `json:"auth"`
	Keepalive  KeepaliveConf  `json:"keepalivetracker"`
	Downloader DownloaderConf `json:"downloader"`
	Dsort      DsortConf      `json:"distributed_sort"`
	Transport  TransportConf  `json:"transport"`
	Memsys     MemsysConf     `json:"memsys"`

	// Transform (offline) or Copy src Bucket => dst bucket
	TCB TCBConf `json:"tcb"`

	// metadata write policy: (immediate | delayed | never)
	WritePolicy WritePolicyConf `json:"write_policy"`

	// standalone enumerated features that can be configured
	// to flip assorted global defaults (see cmn/feat/feat.go)
	Features feat.Flags `json:"features,string" allow:"cluster"`

	// read-only
	LastUpdated string `json:"lastupdate_time"`       // timestamp
	UUID        string `json:"uuid"`                  // UUID
	Version     int64  `json:"config_version,string"` // version
}

global configuration

func (*ClusterConfig) Apply

func (c *ClusterConfig) Apply(updateConf *ConfigToSet, asType string) error

func (*ClusterConfig) JspOpts

func (*ClusterConfig) JspOpts() jsp.Options

func (*ClusterConfig) String

func (c *ClusterConfig) String() string

type Config

type Config struct {
	ClusterConfig `json:",inline"`
	LocalConfig   `json:",inline"`
	// contains filtered or unexported fields
}

Config is a single control structure (persistent and versioned) that contains both cluster (global) and node (local) configuration Naming convention for setting/getting values: (parent section json tag . child json tag) See also: `IterFields`, `IterFieldNameSepa`

func (*Config) SetRole

func (c *Config) SetRole(role string)

func (*Config) TestingEnv

func (c *Config) TestingEnv() bool

TestingEnv returns true if config is set to a development environment where a single local filesystem is partitioned between all (locally running) targets and is used for both local and Cloud buckets See also: `rom.testingEnv`

func (*Config) UpdateClusterConfig

func (c *Config) UpdateClusterConfig(updateConf *ConfigToSet, asType string) (err error)

func (*Config) Validate

func (c *Config) Validate() error

main config validator

type ConfigToSet added in v1.3.21

type ConfigToSet struct {
	// ClusterConfig
	Backend     *BackendConf          `json:"backend,omitempty"`
	Mirror      *MirrorConfToSet      `json:"mirror,omitempty"`
	EC          *ECConfToSet          `json:"ec,omitempty"`
	Log         *LogConfToSet         `json:"log,omitempty"`
	Periodic    *PeriodConfToSet      `json:"periodic,omitempty"`
	Timeout     *TimeoutConfToSet     `json:"timeout,omitempty"`
	Client      *ClientConfToSet      `json:"client,omitempty"`
	Space       *SpaceConfToSet       `json:"space,omitempty"`
	LRU         *LRUConfToSet         `json:"lru,omitempty"`
	Disk        *DiskConfToSet        `json:"disk,omitempty"`
	Rebalance   *RebalanceConfToSet   `json:"rebalance,omitempty"`
	Resilver    *ResilverConfToSet    `json:"resilver,omitempty"`
	Cksum       *CksumConfToSet       `json:"checksum,omitempty"`
	Versioning  *VersionConfToSet     `json:"versioning,omitempty"`
	Net         *NetConfToSet         `json:"net,omitempty"`
	FSHC        *FSHCConfToSet        `json:"fshc,omitempty"`
	Auth        *AuthConfToSet        `json:"auth,omitempty"`
	Keepalive   *KeepaliveConfToSet   `json:"keepalivetracker,omitempty"`
	Downloader  *DownloaderConfToSet  `json:"downloader,omitempty"`
	Dsort       *DsortConfToSet       `json:"distributed_sort,omitempty"`
	Transport   *TransportConfToSet   `json:"transport,omitempty"`
	Memsys      *MemsysConfToSet      `json:"memsys,omitempty"`
	TCB         *TCBConfToSet         `json:"tcb,omitempty"`
	WritePolicy *WritePolicyConfToSet `json:"write_policy,omitempty"`
	Proxy       *ProxyConfToSet       `json:"proxy,omitempty"`
	Features    *feat.Flags           `json:"features,string,omitempty"`

	// LocalConfig
	FSP *FSPConf `json:"fspaths,omitempty"`
}

global configuration

func (*ConfigToSet) FillFromKVS added in v1.3.21

func (ctu *ConfigToSet) FillFromKVS(kvs []string) (err error)

FillFromKVS populates `ConfigToSet` from key value pairs of the form `key=value`

func (*ConfigToSet) FillFromQuery added in v1.3.21

func (ctu *ConfigToSet) FillFromQuery(query url.Values) error

FillFromQuery populates ConfigToSet from URL query values

func (*ConfigToSet) JspOpts added in v1.3.21

func (*ConfigToSet) JspOpts() jsp.Options

func (*ConfigToSet) Merge added in v1.3.21

func (ctu *ConfigToSet) Merge(update *ConfigToSet)

type DiskConf

type DiskConf struct {
	DiskUtilLowWM   int64        `json:"disk_util_low_wm"`  // no throttling below
	DiskUtilHighWM  int64        `json:"disk_util_high_wm"` // throttle longer when above
	DiskUtilMaxWM   int64        `json:"disk_util_max_wm"`
	IostatTimeLong  cos.Duration `json:"iostat_time_long"`
	IostatTimeShort cos.Duration `json:"iostat_time_short"`
}

global configuration

func (*DiskConf) Validate

func (c *DiskConf) Validate() (err error)

type DiskConfToSet added in v1.3.21

type DiskConfToSet struct {
	DiskUtilLowWM   *int64        `json:"disk_util_low_wm,omitempty"`
	DiskUtilHighWM  *int64        `json:"disk_util_high_wm,omitempty"`
	DiskUtilMaxWM   *int64        `json:"disk_util_max_wm,omitempty"`
	IostatTimeLong  *cos.Duration `json:"iostat_time_long,omitempty"`
	IostatTimeShort *cos.Duration `json:"iostat_time_short,omitempty"`
}

global configuration

type DownloaderConf

type DownloaderConf struct {
	Timeout cos.Duration `json:"timeout"`
}

global configuration

func (*DownloaderConf) Validate

func (c *DownloaderConf) Validate() error

type DownloaderConfToSet added in v1.3.21

type DownloaderConfToSet struct {
	Timeout *cos.Duration `json:"timeout,omitempty"`
}

global configuration

type DsortConf added in v1.3.21

type DsortConf struct {
	DuplicatedRecords   string       `json:"duplicated_records"`
	MissingShards       string       `json:"missing_shards"` // cmn.SupportedReactions enum
	EKMMalformedLine    string       `json:"ekm_malformed_line"`
	EKMMissingKey       string       `json:"ekm_missing_key"`
	DefaultMaxMemUsage  string       `json:"default_max_mem_usage"`
	CallTimeout         cos.Duration `json:"call_timeout"`
	DsorterMemThreshold string       `json:"dsorter_mem_threshold"`
	Compression         string       `json:"compression"`       // {CompressAlways,...} in api/apc/compression.go
	SbundleMult         int          `json:"bundle_multiplier"` // stream-bundle multiplier: num to destination
}

global configuration

func (*DsortConf) Validate added in v1.3.21

func (c *DsortConf) Validate() (err error)

func (*DsortConf) ValidateWithOpts added in v1.3.21

func (c *DsortConf) ValidateWithOpts(allowEmpty bool) (err error)

type DsortConfToSet added in v1.3.21

type DsortConfToSet struct {
	DuplicatedRecords   *string       `json:"duplicated_records,omitempty"`
	MissingShards       *string       `json:"missing_shards,omitempty"`
	EKMMalformedLine    *string       `json:"ekm_malformed_line,omitempty"`
	EKMMissingKey       *string       `json:"ekm_missing_key,omitempty"`
	DefaultMaxMemUsage  *string       `json:"default_max_mem_usage,omitempty"`
	CallTimeout         *cos.Duration `json:"call_timeout,omitempty"`
	DsorterMemThreshold *string       `json:"dsorter_mem_threshold,omitempty"`
	Compression         *string       `json:"compression,omitempty"`
	SbundleMult         *int          `json:"bundle_multiplier,omitempty"`
}

global configuration

type ECConf

type ECConf struct {
	Compression string `json:"compression"` // enum { CompressAlways, ... } in api/apc/compression.go

	// ObjSizeLimit is object size threshold _separating_ intra-cluster mirroring from
	// erasure coding.
	//
	// The value 0 (zero) indicates that objects of any size
	// are to be sliced, to produce (D) data slices and (P) erasure coded parity slices.
	// On the other hand, the value -1 specifies that absolutely all objects of any size
	// must be replicated as is. In effect, the (-1) option provides data protection via
	// intra-cluster (P+1)-way replication (a.k.a. mirroring).
	//
	// In all cases, a given (D, P) configuration provides node-level redundancy,
	// whereby P nodes can be lost without incurring loss of data.
	ObjSizeLimit int64 `json:"objsize_limit"`

	// Number of data (D) slices; the value 1 will have an effect of producing
	// (P) additional full-size replicas.
	DataSlices int `json:"data_slices"`

	// Depending on the object size and `ObjSizeLimit`, the value of `ParitySlices` (or P) indicates:
	// - a number of additional parity slices (generated or _computed_ from the (D) data slices),
	// or:
	// - a number of full object replicas (copies).
	// In all cases, the same rule applies: all slices and/or all full copies are stored on different
	// storage nodes (a.k.a. targets).
	ParitySlices int `json:"parity_slices"`

	SbundleMult int `json:"bundle_multiplier"` // stream-bundle multiplier: num streams to destination

	Enabled  bool `json:"enabled"`   // EC is enabled
	DiskOnly bool `json:"disk_only"` // if true, EC does not use SGL - data goes directly to drives
}

global configuration

func (*ECConf) RequiredRestoreTargets

func (c *ECConf) RequiredRestoreTargets() int

func (*ECConf) String

func (c *ECConf) String() string

func (*ECConf) Validate

func (c *ECConf) Validate() error

func (*ECConf) ValidateAsProps

func (c *ECConf) ValidateAsProps(arg ...any) (err error)

type ECConfToSet added in v1.3.21

type ECConfToSet struct {
	ObjSizeLimit *int64  `json:"objsize_limit,omitempty"`
	Compression  *string `json:"compression,omitempty"`
	SbundleMult  *int    `json:"bundle_multiplier,omitempty"`
	DataSlices   *int    `json:"data_slices,omitempty"`
	ParitySlices *int    `json:"parity_slices,omitempty"`
	Enabled      *bool   `json:"enabled,omitempty"`
	DiskOnly     *bool   `json:"disk_only,omitempty"`
}

global configuration

type ETLErrCtx added in v1.3.16

type ETLErrCtx struct {
	TID     string
	ETLName string
	PodName string
	SvcName string
}

assorted aistore errors

type ErrAborted

type ErrAborted struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func AsErrAborted

func AsErrAborted(err error) (errAborted *ErrAborted)

func NewErrAborted

func NewErrAborted(what, ctx string, err error) *ErrAborted

func (*ErrAborted) Error

func (e *ErrAborted) Error() (s string)

func (*ErrAborted) Unwrap

func (e *ErrAborted) Unwrap() (err error)

type ErrBckNotFound

type ErrBckNotFound struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrBckNotFound

func NewErrBckNotFound(bck *Bck) *ErrBckNotFound

func (*ErrBckNotFound) Error

func (e *ErrBckNotFound) Error() string

type ErrBucketAccessDenied

type ErrBucketAccessDenied struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewBucketAccessDenied

func NewBucketAccessDenied(bucket, oper string, aattrs apc.AccessAttrs) *ErrBucketAccessDenied

func (*ErrBucketAccessDenied) Error

func (e *ErrBucketAccessDenied) Error() string

func (*ErrBucketAccessDenied) String

func (e *ErrBucketAccessDenied) String() string

type ErrBucketAlreadyExists

type ErrBucketAlreadyExists struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrBckAlreadyExists

func NewErrBckAlreadyExists(bck *Bck) *ErrBucketAlreadyExists

func (*ErrBucketAlreadyExists) Error

func (e *ErrBucketAlreadyExists) Error() string

type ErrBusy added in v1.3.21

type ErrBusy struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrBusy added in v1.3.21

func NewErrBusy(what string, name fmt.Stringer, detail string) *ErrBusy

func (*ErrBusy) Error added in v1.3.21

func (e *ErrBusy) Error() string

type ErrCapExceeded added in v1.3.19

type ErrCapExceeded struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrCapExceeded added in v1.3.19

func NewErrCapExceeded(totalBytesUsed, totalBytes uint64, highWM, cleanupWM int64, usedPct int32, oos bool) *ErrCapExceeded

func (*ErrCapExceeded) Error added in v1.3.19

func (e *ErrCapExceeded) Error() string

type ErrETL

type ErrETL struct {
	Reason string
	ETLErrCtx
}

assorted aistore errors

func NewErrETL

func NewErrETL(ctx *ETLErrCtx, format string, a ...any) *ErrETL

func (*ErrETL) Error

func (e *ErrETL) Error() string

func (*ErrETL) WithContext

func (e *ErrETL) WithContext(ctx *ETLErrCtx) *ErrETL

func (*ErrETL) WithPodName

func (e *ErrETL) WithPodName(name string) *ErrETL

type ErrFailedTo

type ErrFailedTo struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrFailedTo

func NewErrFailedTo(actor fmt.Stringer, action string, what any, err error, errCode ...int) *ErrFailedTo

func (*ErrFailedTo) Error

func (e *ErrFailedTo) Error() string

func (*ErrFailedTo) Unwrap

func (e *ErrFailedTo) Unwrap() (err error)

type ErrHTTP

type ErrHTTP struct {
	TypeCode   string `json:"tcode,omitempty"`
	Message    string `json:"message"`
	Method     string `json:"method"`
	URLPath    string `json:"url_path"`
	RemoteAddr string `json:"remote_addr"`
	Caller     string `json:"caller"`
	Node       string `json:"node"`

	Status int `json:"status"`
	// contains filtered or unexported fields
}

API error structure is returned to aistore client and carries one of the specific errors enumerated below

func Err2HTTPErr

func Err2HTTPErr(err error) *ErrHTTP

func InitErrHTTP

func InitErrHTTP(r *http.Request, err error, errCode int) (e *ErrHTTP)

uses `allocHterr` to allocate - caller must free via `FreeHterr`

func NewErrHTTP

func NewErrHTTP(r *http.Request, err error, errCode int) (e *ErrHTTP)

func Str2HTTPErr added in v1.3.18

func Str2HTTPErr(msg string) *ErrHTTP

func (*ErrHTTP) Error

func (e *ErrHTTP) Error() (s string)

func (*ErrHTTP) StringEx added in v1.3.18

func (e *ErrHTTP) StringEx() (s string)

Example: ErrBckNotFound: bucket "ais://abc" does not exist: HEAD /v1/buckets/abc (p[kWQp8080]: htrun.go:1035 <- prxtrybck.go:180 <- ...

type ErrInitBackend

type ErrInitBackend struct {
	Provider string
}

assorted aistore errors

func (*ErrInitBackend) Error

func (e *ErrInitBackend) Error() string

type ErrInvalidBackendProvider

type ErrInvalidBackendProvider struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func (*ErrInvalidBackendProvider) Error

func (e *ErrInvalidBackendProvider) Error() string

func (*ErrInvalidBackendProvider) Is

func (*ErrInvalidBackendProvider) Is(target error) bool

type ErrInvalidCksum

type ErrInvalidCksum struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrInvalidCksum

func NewErrInvalidCksum(eHash, aHash string) *ErrInvalidCksum

func (*ErrInvalidCksum) Error

func (e *ErrInvalidCksum) Error() string

func (*ErrInvalidCksum) Expected

func (e *ErrInvalidCksum) Expected() string

type ErrInvalidFSPathsConf

type ErrInvalidFSPathsConf struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrInvalidFSPathsConf

func NewErrInvalidFSPathsConf(err error) *ErrInvalidFSPathsConf

func (*ErrInvalidFSPathsConf) Error

func (e *ErrInvalidFSPathsConf) Error() string

func (*ErrInvalidFSPathsConf) Unwrap

func (e *ErrInvalidFSPathsConf) Unwrap() (err error)

type ErrInvalidMountpath

type ErrInvalidMountpath struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrInvalidaMountpath

func NewErrInvalidaMountpath(mpath, cause string) *ErrInvalidMountpath

func (*ErrInvalidMountpath) Error

func (e *ErrInvalidMountpath) Error() string

type ErrInvalidObjName added in v1.3.22

type ErrInvalidObjName struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func ValidateObjName added in v1.3.22

func ValidateObjName(name string) (err *ErrInvalidObjName)

func (*ErrInvalidObjName) Error added in v1.3.22

func (e *ErrInvalidObjName) Error() string

type ErrLimitedCoexistence

type ErrLimitedCoexistence struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrLimitedCoexistence

func NewErrLimitedCoexistence(node, xaction, action, detail string) *ErrLimitedCoexistence

func (*ErrLimitedCoexistence) Error

func (e *ErrLimitedCoexistence) Error() string

type ErrLmetaCorrupted

type ErrLmetaCorrupted struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrLmetaCorrupted

func NewErrLmetaCorrupted(err error) *ErrLmetaCorrupted

func (*ErrLmetaCorrupted) Error

func (e *ErrLmetaCorrupted) Error() string

func (*ErrLmetaCorrupted) Unwrap

func (e *ErrLmetaCorrupted) Unwrap() (err error)

type ErrLmetaNotFound

type ErrLmetaNotFound struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrLmetaNotFound

func NewErrLmetaNotFound(err error) *ErrLmetaNotFound

func (*ErrLmetaNotFound) Error

func (e *ErrLmetaNotFound) Error() string

func (*ErrLmetaNotFound) Unwrap

func (e *ErrLmetaNotFound) Unwrap() (err error)

type ErrMissingBackend

type ErrMissingBackend struct {
	Provider string
	Msg      string
}

assorted aistore errors

func (*ErrMissingBackend) Error

func (e *ErrMissingBackend) Error() string

type ErrMountpathNotFound

type ErrMountpathNotFound struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrMountpathNotFound

func NewErrMountpathNotFound(mpath, fqn string, disabled bool) *ErrMountpathNotFound

func (*ErrMountpathNotFound) Error

func (e *ErrMountpathNotFound) Error() string

type ErrNoNodes

type ErrNoNodes struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrNoNodes

func NewErrNoNodes(role string, mmcount int) *ErrNoNodes

func (*ErrNoNodes) Error

func (e *ErrNoNodes) Error() (s string)

type ErrNotImpl

type ErrNotImpl struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrNotImpl

func NewErrNotImpl(action, what string) *ErrNotImpl

func (*ErrNotImpl) Error

func (e *ErrNotImpl) Error() string

type ErrNotRemoteBck added in v1.3.22

type ErrNotRemoteBck struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func ValidateRemoteBck added in v1.3.22

func ValidateRemoteBck(act string, bck *Bck) (err *ErrNotRemoteBck)

func (*ErrNotRemoteBck) Error added in v1.3.22

func (e *ErrNotRemoteBck) Error() string

type ErrObjDefunct

type ErrObjDefunct struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrObjDefunct

func NewErrObjDefunct(name string, d1, d2 uint64) *ErrObjDefunct

func (*ErrObjDefunct) Error

func (e *ErrObjDefunct) Error() string

type ErrObjectAccessDenied

type ErrObjectAccessDenied struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewObjectAccessDenied

func NewObjectAccessDenied(object, oper string, aattrs apc.AccessAttrs) *ErrObjectAccessDenied

func (*ErrObjectAccessDenied) Error

func (e *ErrObjectAccessDenied) Error() string

func (*ErrObjectAccessDenied) String

func (e *ErrObjectAccessDenied) String() string

type ErrRangeNotSatisfiable added in v1.3.22

type ErrRangeNotSatisfiable struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrRangeNotSatisfiable added in v1.3.22

func NewErrRangeNotSatisfiable(err error, ranges []string, size int64) *ErrRangeNotSatisfiable

func (*ErrRangeNotSatisfiable) Error added in v1.3.22

func (e *ErrRangeNotSatisfiable) Error() string

type ErrRemoteBckNotFound

type ErrRemoteBckNotFound struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrRemoteBckNotFound

func NewErrRemoteBckNotFound(bck *Bck) *ErrRemoteBckNotFound

func (*ErrRemoteBckNotFound) Error

func (e *ErrRemoteBckNotFound) Error() string

type ErrRemoteBucketOffline

type ErrRemoteBucketOffline struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrRemoteBckOffline

func NewErrRemoteBckOffline(bck *Bck) *ErrRemoteBucketOffline

func (*ErrRemoteBucketOffline) Error

func (e *ErrRemoteBucketOffline) Error() string

type ErrSoft

type ErrSoft struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrSoft

func NewErrSoft(what string) *ErrSoft

func (*ErrSoft) Error

func (e *ErrSoft) Error() string

type ErrStreamTerminated added in v1.3.16

type ErrStreamTerminated struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrStreamTerminated added in v1.3.16

func NewErrStreamTerminated(stream string, err error, reason, detail string) *ErrStreamTerminated

func (*ErrStreamTerminated) Error added in v1.3.16

func (e *ErrStreamTerminated) Error() string

func (*ErrStreamTerminated) Unwrap added in v1.3.16

func (e *ErrStreamTerminated) Unwrap() (err error)

type ErrUnsupp

type ErrUnsupp struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrUnsupp

func NewErrUnsupp(action, what string) *ErrUnsupp

func (*ErrUnsupp) Error

func (e *ErrUnsupp) Error() string

type ErrXactNotFound added in v1.3.16

type ErrXactNotFound struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrXactNotFoundError

func NewErrXactNotFoundError(cause string) *ErrXactNotFound

func (*ErrXactNotFound) Error added in v1.3.16

func (e *ErrXactNotFound) Error() string

type ErrXactTgtInMaint added in v1.3.18

type ErrXactTgtInMaint struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrXactTgtInMaint added in v1.3.18

func NewErrXactTgtInMaint(xaction, tname string) *ErrXactTgtInMaint

func (*ErrXactTgtInMaint) Error added in v1.3.18

func (e *ErrXactTgtInMaint) Error() string

type ErrXactUsePrev added in v1.3.18

type ErrXactUsePrev struct {
	// contains filtered or unexported fields
}

assorted aistore errors

func NewErrXactUsePrev added in v1.3.18

func NewErrXactUsePrev(xaction string) *ErrXactUsePrev

func (*ErrXactUsePrev) Error added in v1.3.18

func (e *ErrXactUsePrev) Error() string

type ExtraProps

type ExtraProps struct {
	AWS  ExtraPropsAWS  `json:"aws,omitempty" list:"omitempty"`
	HTTP ExtraPropsHTTP `json:"http,omitempty" list:"omitempty"`
	HDFS ExtraPropsHDFS `json:"hdfs,omitempty" list:"omitempty"`
}

func (*ExtraProps) ValidateAsProps

func (c *ExtraProps) ValidateAsProps(arg ...any) error

type ExtraPropsAWS

type ExtraPropsAWS struct {
	CloudRegion string `json:"cloud_region,omitempty"`

	// from https://github.com/aws/aws-sdk-go/blob/main/aws/config.go:
	// - "An optional endpoint URL (hostname only or fully qualified URI)
	// that overrides the default generated endpoint."
	Endpoint string `json:"endpoint,omitempty"`

	// from https://github.com/aws/aws-sdk-go/blob/main/aws/session/session.go:
	// - "Overrides the config profile the Session should be created from. If not
	// set the value of the environment variable will be loaded (AWS_PROFILE,
	// or AWS_DEFAULT_PROFILE if the Shared Config is enabled)."
	Profile string `json:"profile,omitempty"`

	// Amazon S3: 1000
	// - https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-pagination.html#cli-usage-pagination-serverside
	// vs OpenStack Swift: 10,000
	// - https://docs.openstack.org/swift/latest/api/pagination.html
	MaxPageSize uint `json:"max_pagesize,omitempty"`
}

type ExtraPropsAWSToSet added in v1.3.21

type ExtraPropsAWSToSet struct {
	CloudRegion *string `json:"cloud_region"`
	Endpoint    *string `json:"endpoint"`
	Profile     *string `json:"profile"`
	MaxPageSize *uint   `json:"max_pagesize"`
}

type ExtraPropsHDFS

type ExtraPropsHDFS struct {
	// Reference directory.
	RefDirectory string `json:"ref_directory,omitempty"`
}

type ExtraPropsHDFSToSet added in v1.3.21

type ExtraPropsHDFSToSet struct {
	RefDirectory *string `json:"ref_directory"`
}

type ExtraPropsHTTP

type ExtraPropsHTTP struct {
	// Original URL prior to hashing.
	OrigURLBck string `json:"original_url,omitempty" list:"readonly"`
}

type ExtraPropsHTTPToSet added in v1.3.21

type ExtraPropsHTTPToSet struct {
	OrigURLBck *string `json:"original_url"`
}

type ExtraToSet added in v1.3.21

type ExtraToSet struct {
	AWS  *ExtraPropsAWSToSet  `json:"aws"`
	HTTP *ExtraPropsHTTPToSet `json:"http"`
	HDFS *ExtraPropsHDFSToSet `json:"hdfs"`
}

type FSHCConf

type FSHCConf struct {
	TestFileCount int  `json:"test_files"`  // number of files to read/write
	ErrorLimit    int  `json:"error_limit"` // exceeding err limit causes disabling mountpath
	Enabled       bool `json:"enabled"`
}

global configuration

type FSHCConfToSet added in v1.3.21

type FSHCConfToSet struct {
	TestFileCount *int  `json:"test_files,omitempty"`
	ErrorLimit    *int  `json:"error_limit,omitempty"`
	Enabled       *bool `json:"enabled,omitempty"`
}

global configuration

type FSPConf

type FSPConf struct {
	Paths cos.StrSet `json:"paths,omitempty" list:"readonly"`
}

ais node: fspaths (a.k.a. mountpaths)

func (*FSPConf) MarshalJSON

func (c *FSPConf) MarshalJSON() (data []byte, err error)

func (*FSPConf) UnmarshalJSON

func (c *FSPConf) UnmarshalJSON(data []byte) (err error)

func (*FSPConf) Validate

func (c *FSPConf) Validate(contextConfig *Config) error

type HTTPBckObj

type HTTPBckObj struct {
	Bck        Bck
	ObjName    string
	OrigURLBck string // HTTP URL of the bucket (object name excluded)
}

Represents the AIS bucket, object and URL associated with a HTTP resource

func NewHTTPObj

func NewHTTPObj(u *url.URL) *HTTPBckObj

func NewHTTPObjPath

func NewHTTPObjPath(rawURL string) (*HTTPBckObj, error)

type HTTPConf

type HTTPConf struct {
	Proto           string `json:"-"`                 // http or https (set depending on `UseHTTPS`)
	Certificate     string `json:"server_crt"`        // HTTPS: X509 certificate
	CertKey         string `json:"server_key"`        // HTTPS: X509 key
	ServerNameTLS   string `json:"domain_tls"`        // #6410
	ClientCA        string `json:"client_ca_tls"`     // #6410
	ClientAuthTLS   int    `json:"client_auth_tls"`   // #6410 tls.ClientAuthType enum
	WriteBufferSize int    `json:"write_buffer_size"` // http.Transport.WriteBufferSize; zero defaults to 4KB
	ReadBufferSize  int    `json:"read_buffer_size"`  // http.Transport.ReadBufferSize; ditto
	UseHTTPS        bool   `json:"use_https"`         // use HTTPS
	SkipVerifyCrt   bool   `json:"skip_verify"`       // skip X509 cert verification (used with self-signed certs)
	Chunked         bool   `json:"chunked_transfer"`  // (https://tools.ietf.org/html/rfc7230#page-36; not used since 02/23)
}

global configuration

func (*HTTPConf) ToTLS added in v1.3.21

func (c *HTTPConf) ToTLS() TLSArgs

used intra-clients; see related: EnvToTLS()

func (*HTTPConf) Validate added in v1.3.22

func (c *HTTPConf) Validate() error

type HTTPConfToSet added in v1.3.21

type HTTPConfToSet struct {
	Certificate     *string `json:"server_crt,omitempty"`
	CertKey         *string `json:"server_key,omitempty"`
	ServerNameTLS   *string `json:"domain_tls,omitempty"`
	ClientCA        *string `json:"client_ca_tls,omitempty"`
	WriteBufferSize *int    `json:"write_buffer_size,omitempty" list:"readonly"`
	ReadBufferSize  *int    `json:"read_buffer_size,omitempty" list:"readonly"`
	ClientAuthTLS   *int    `json:"client_auth_tls,omitempty"`
	UseHTTPS        *bool   `json:"use_https,omitempty"`
	SkipVerifyCrt   *bool   `json:"skip_verify,omitempty"`
	Chunked         *bool   `json:"chunked_transfer,omitempty"`
}

global configuration

type HreqArgs

type HreqArgs struct {
	BodyR    io.Reader
	Header   http.Header // request headers
	Query    url.Values  // query, e.g. ?a=x&b=y&c=z
	RawQuery string      // raw query
	Method   string
	Base     string // base URL, e.g. http://xyz.abc
	Path     string // path URL, e.g. /x/y/z
	Body     []byte
}

usage 1: initialize and fill out HTTP request. usage 2: intra-cluster control-plane (except streams) usage 3: PUT and APPEND API BodyR optimizes-out allocations - if non-nil and implements `io.Closer`, will always be closed by `client.Do`

func AllocHra

func AllocHra() (a *HreqArgs)

func (*HreqArgs) Req

func (u *HreqArgs) Req() (*http.Request, error)

func (*HreqArgs) ReqWithCancel

func (u *HreqArgs) ReqWithCancel() (*http.Request, context.Context, context.CancelFunc, error)

ReqWithCancel creates request with ability to cancel it.

func (*HreqArgs) ReqWithTimeout

func (u *HreqArgs) ReqWithTimeout(timeout time.Duration) (*http.Request, context.Context, context.CancelFunc, error)

func (*HreqArgs) URL

func (u *HreqArgs) URL() string

type IterField

type IterField interface {
	Value() any                          // returns the value
	String() string                      // string representation of the value
	SetValue(v any, force ...bool) error // `force` ignores `tagReadonly` (to be used with caution!)
}

Represents a single named field

type IterOpts

type IterOpts struct {
	// Skip fields based on allowed tag
	Allowed string
	// Visits all the fields, not only the leaves.
	VisitAll bool
	// Read-only walk is true by default (compare with `UpdateFieldValue`)
	// Note that `tagOmitempty` is limited to read-only - has no effect when `OnlyRead == false`.
	OnlyRead bool
}

type KeepaliveConf

type KeepaliveConf struct {
	Proxy       KeepaliveTrackerConf `json:"proxy"`  // how proxy tracks target keepalives
	Target      KeepaliveTrackerConf `json:"target"` // how target tracks primary proxies keepalives
	RetryFactor uint8                `json:"retry_factor"`
}

global configuration

func (*KeepaliveConf) Validate

func (c *KeepaliveConf) Validate() (err error)

type KeepaliveConfToSet added in v1.3.21

type KeepaliveConfToSet struct {
	Proxy       *KeepaliveTrackerConfToSet `json:"proxy,omitempty"`
	Target      *KeepaliveTrackerConfToSet `json:"target,omitempty"`
	RetryFactor *uint8                     `json:"retry_factor,omitempty"`
}

global configuration

type KeepaliveTrackerConf

type KeepaliveTrackerConf struct {
	Name     string       `json:"name"`     // "heartbeat" (other enumerated values TBD)
	Interval cos.Duration `json:"interval"` // keepalive interval
	Factor   uint8        `json:"factor"`   // only average
}

keepalive tracker

type KeepaliveTrackerConfToSet added in v1.3.21

type KeepaliveTrackerConfToSet struct {
	Interval *cos.Duration `json:"interval,omitempty"`
	Name     *string       `json:"name,omitempty" list:"readonly"`
	Factor   *uint8        `json:"factor,omitempty"`
}

global configuration

type L4Conf

type L4Conf struct {
	Proto         string `json:"proto"`           // tcp, udp
	SndRcvBufSize int    `json:"sndrcv_buf_size"` // SO_RCVBUF and SO_SNDBUF
}

global configuration

type LRUConf

type LRUConf struct {
	// DontEvictTimeStr denotes the period of time during which eviction of an object
	// is forbidden [atime, atime + DontEvictTime]
	DontEvictTime cos.Duration `json:"dont_evict_time"`

	// CapacityUpdTimeStr denotes the frequency at which AIStore updates local capacity utilization
	CapacityUpdTime cos.Duration `json:"capacity_upd_time"`

	// Enabled: LRU will only run when set to true
	Enabled bool `json:"enabled"`
}

global configuration

func (*LRUConf) String

func (c *LRUConf) String() string

func (*LRUConf) Validate

func (c *LRUConf) Validate() (err error)

type LRUConfToSet added in v1.3.21

type LRUConfToSet struct {
	DontEvictTime   *cos.Duration `json:"dont_evict_time,omitempty"`
	CapacityUpdTime *cos.Duration `json:"capacity_upd_time,omitempty"`
	Enabled         *bool         `json:"enabled,omitempty"`
}

global configuration

type LocalConfig

type LocalConfig struct {
	ConfigDir string         `json:"confdir"`
	LogDir    string         `json:"log_dir"`
	HostNet   LocalNetConfig `json:"host_net"`
	FSP       FSPConf        `json:"fspaths"`
	TestFSP   TestFSPConf    `json:"test_fspaths"`
}

local config

func (*LocalConfig) AddPath

func (c *LocalConfig) AddPath(mpath string)

func (*LocalConfig) DelPath

func (c *LocalConfig) DelPath(mpath string)

func (*LocalConfig) JspOpts

func (*LocalConfig) JspOpts() jsp.Options

func (*LocalConfig) TestingEnv

func (c *LocalConfig) TestingEnv() bool

type LocalNetConfig

type LocalNetConfig struct {
	Hostname             string `json:"hostname"`
	HostnameIntraControl string `json:"hostname_intra_control"`
	HostnameIntraData    string `json:"hostname_intra_data"`
	Port                 int    `json:"port,string"`               // listening port
	PortIntraControl     int    `json:"port_intra_control,string"` // --/-- for intra-cluster control
	PortIntraData        int    `json:"port_intra_data,string"`    // --/-- for intra-cluster data
	// omit
	UseIntraControl bool `json:"-"`
	UseIntraData    bool `json:"-"`
}

ais node: (local) network config

func (*LocalNetConfig) Validate

func (c *LocalNetConfig) Validate(contextConfig *Config) (err error)

type LogConf

type LogConf struct {
	Level     cos.LogLevel `json:"level"`      // log level (aka verbosity)
	MaxSize   cos.SizeIEC  `json:"max_size"`   // exceeding this size triggers log rotation
	MaxTotal  cos.SizeIEC  `json:"max_total"`  // (sum individual log sizes); exceeding this number triggers cleanup
	FlushTime cos.Duration `json:"flush_time"` // log flush interval
	StatsTime cos.Duration `json:"stats_time"` // (not used)
}

global configuration

func (*LogConf) Validate

func (c *LogConf) Validate() error

type LogConfToSet added in v1.3.21

type LogConfToSet struct {
	Level     *cos.LogLevel `json:"level,omitempty"`
	MaxSize   *cos.SizeIEC  `json:"max_size,omitempty"`
	MaxTotal  *cos.SizeIEC  `json:"max_total,omitempty"`
	FlushTime *cos.Duration `json:"flush_time,omitempty"`
	StatsTime *cos.Duration `json:"stats_time,omitempty"`
}

global configuration

type LsoEntries

type LsoEntries []*LsoEntry

LsoEntry is a single entry in LsoResult.Entries slice (below) containing list-objects result for the corresponding (listed) object or an archived file; `Flags` is a bit field where `EntryStatusBits` bits [0-4] are reserved for object status (all statuses are mutually exclusive)

func (*LsoEntries) DecodeMsg added in v1.3.22

func (z *LsoEntries) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (LsoEntries) EncodeMsg added in v1.3.22

func (z LsoEntries) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (LsoEntries) Msgsize added in v1.3.22

func (z LsoEntries) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

type LsoEntry

type LsoEntry struct {
	Name     string `json:"name" msg:"n"`                            // object name
	Checksum string `json:"checksum,omitempty" msg:"cs,omitempty"`   // checksum
	Atime    string `json:"atime,omitempty" msg:"a,omitempty"`       // last access time; formatted as ListObjsMsg.TimeFormat
	Version  string `json:"version,omitempty" msg:"v,omitempty"`     // e.g., GCP int64 generation, AWS version (string), etc.
	Location string `json:"location,omitempty" msg:"t,omitempty"`    // [tnode:mountpath]
	Custom   string `json:"custom-md,omitempty" msg:"m,omitempty"`   // custom metadata: ETag, MD5, CRC, user-defined ...
	Size     int64  `json:"size,string,omitempty" msg:"s,omitempty"` // size in bytes
	Copies   int16  `json:"copies,omitempty" msg:"c,omitempty"`      // ## copies (NOTE: for non-replicated object copies == 1)
	Flags    uint16 `json:"flags,omitempty" msg:"f,omitempty"`       // enum { EntryIsCached, EntryIsDir, EntryInArch, ...}
}

LsoEntry is a single entry in LsoResult.Entries slice (below) containing list-objects result for the corresponding (listed) object or an archived file; `Flags` is a bit field where `EntryStatusBits` bits [0-4] are reserved for object status (all statuses are mutually exclusive)

func DedupLso added in v1.3.16

func DedupLso(entries LsoEntries, maxSize int) []*LsoEntry

func (*LsoEntry) CopyWithProps

func (be *LsoEntry) CopyWithProps(propsSet cos.StrSet) (ne *LsoEntry)

func (*LsoEntry) DecodeMsg

func (z *LsoEntry) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*LsoEntry) EncodeMsg

func (z *LsoEntry) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*LsoEntry) IsInsideArch

func (be *LsoEntry) IsInsideArch() bool

func (*LsoEntry) IsListedArch added in v1.3.18

func (be *LsoEntry) IsListedArch() bool

func (*LsoEntry) IsPresent added in v1.3.21

func (be *LsoEntry) IsPresent() bool

The terms "cached" and "present" are interchangeable: "object is cached" and "is present" is actually the same thing

func (*LsoEntry) IsStatusOK

func (be *LsoEntry) IsStatusOK() bool

func (*LsoEntry) IsVerChanged added in v1.3.22

func (be *LsoEntry) IsVerChanged() bool

func (*LsoEntry) IsVerRemoved added in v1.3.22

func (be *LsoEntry) IsVerRemoved() bool

func (*LsoEntry) Msgsize

func (z *LsoEntry) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

func (*LsoEntry) SetPresent

func (be *LsoEntry) SetPresent()

func (*LsoEntry) SetVerChanged added in v1.3.22

func (be *LsoEntry) SetVerChanged()

see also: "latest-ver", QparamLatestVer, et al.

func (*LsoEntry) SetVerRemoved added in v1.3.22

func (be *LsoEntry) SetVerRemoved()

func (*LsoEntry) Status

func (be *LsoEntry) Status() uint16

func (*LsoEntry) String

func (be *LsoEntry) String() string

type LsoResult

type LsoResult struct {
	UUID              string     `json:"uuid"`
	ContinuationToken string     `json:"continuation_token"`
	Entries           LsoEntries `json:"entries"`
	Flags             uint32     `json:"flags"`
}

LsoResult carries the results of `api.ListObjects`, `BackendProvider.ListObjects`, and friends

func MergeLso

func MergeLso(lists []*LsoResult, maxSize int) *LsoResult

MergeLso merges list-objects results received from targets. For the same object name (ie., the same object) the corresponding properties are merged. If maxSize is greater than 0, the resulting list is sorted and truncated.

func (*LsoResult) DecodeMsg

func (z *LsoResult) DecodeMsg(dc *msgp.Reader) (err error)

DecodeMsg implements msgp.Decodable

func (*LsoResult) EncodeMsg

func (z *LsoResult) EncodeMsg(en *msgp.Writer) (err error)

EncodeMsg implements msgp.Encodable

func (*LsoResult) Msgsize

func (z *LsoResult) Msgsize() (s int)

Msgsize returns an upper bound estimate of the number of bytes occupied by the serialized message

type MemsysConf

type MemsysConf struct {
	MinFree        cos.SizeIEC  `json:"min_free"`
	DefaultBufSize cos.SizeIEC  `json:"default_buf"`
	SizeToGC       cos.SizeIEC  `json:"to_gc"`
	HousekeepTime  cos.Duration `json:"hk_time"`
	MinPctTotal    int          `json:"min_pct_total"`
	MinPctFree     int          `json:"min_pct_free"`
}

global configuration

func (*MemsysConf) Validate

func (c *MemsysConf) Validate() (err error)

type MemsysConfToSet added in v1.3.21

type MemsysConfToSet struct {
	MinFree        *cos.SizeIEC  `json:"min_free,omitempty"`
	DefaultBufSize *cos.SizeIEC  `json:"default_buf,omitempty"`
	SizeToGC       *cos.SizeIEC  `json:"to_gc,omitempty"`
	HousekeepTime  *cos.Duration `json:"hk_time,omitempty"`
	MinPctTotal    *int          `json:"min_pct_total,omitempty"`
	MinPctFree     *int          `json:"min_pct_free,omitempty"`
}

global configuration

type MirrorConf

type MirrorConf struct {
	Copies  int64 `json:"copies"`       // num copies
	Burst   int   `json:"burst_buffer"` // xaction channel (buffer) size
	Enabled bool  `json:"enabled"`      // enabled (to generate copies)
}

global configuration

func (*MirrorConf) String

func (c *MirrorConf) String() string

func (*MirrorConf) Validate

func (c *MirrorConf) Validate() error

func (*MirrorConf) ValidateAsProps

func (c *MirrorConf) ValidateAsProps(...any) error

type MirrorConfToSet added in v1.3.21

type MirrorConfToSet struct {
	Copies  *int64 `json:"copies,omitempty"`
	Burst   *int   `json:"burst_buffer,omitempty"`
	Enabled *bool  `json:"enabled,omitempty"`
}

global configuration

type NetConf

type NetConf struct {
	L4   L4Conf   `json:"l4"`
	HTTP HTTPConf `json:"http"`
}

global configuration

func (*NetConf) Validate

func (c *NetConf) Validate() (err error)

type NetConfToSet added in v1.3.21

type NetConfToSet struct {
	HTTP *HTTPConfToSet `json:"http,omitempty"`
}

global configuration

type Ns

type Ns struct {
	// UUID of other remote AIS cluster (for now only used for AIS). Note
	// that we can have different namespaces which refer to same UUID (cluster).
	// This means that in a sense UUID is a parent of the actual namespace.
	UUID string `json:"uuid" yaml:"uuid"`
	// Name uniquely identifies a namespace under the same UUID (which may
	// be empty) and is used in building FQN for the objects.
	Name string `json:"name" yaml:"name"`
}

Ns (or Namespace) adds additional layer for scoping the data under the same provider. It allows to have same dataset and bucket names under different namespaces what allows for easy data manipulation without affecting data in different namespaces.

func ParseNsUname

func ParseNsUname(s string) (n Ns)

Parses [@uuid][#namespace]. It does a little bit more than just parsing a string from `Uname` so that logic can be reused in different places.

func (Ns) IsAnyRemote

func (n Ns) IsAnyRemote() bool

func (Ns) IsGlobal

func (n Ns) IsGlobal() bool

func (Ns) IsRemote

func (n Ns) IsRemote() bool

func (Ns) Len added in v1.3.21

func (n Ns) Len() int

func (Ns) String

func (n Ns) String() (res string)

func (Ns) Uname

func (n Ns) Uname() string

type OWT

type OWT int

Object Write Transaction (OWT) controls certain aspects of creating new objects in the cluster. In particular, OwtGet* group below simultaneously specifies cold-GET variations (that all involve reading from a remote backend) and the associated locking (that will always reflect a tradeoff between consistency and parallelism) NOTE: enum entries' order is important

const (
	// PUT and PUT-like transactions
	OwtPut       OWT = iota // PUT
	OwtPromote              // promote target-accessible files and directories
	OwtArchive              // multi-obj arch
	OwtTransform            // ETL
	OwtCopy                 // copy and move objects within cluster
	OwtRebalance            // NOTE: must be the last in PUT* group
	//
	// GET and friends
	//
	OwtGetTryLock      // if !try-lock(exclusive) { return error }; read from remote; ...
	OwtGetLock         // lock(exclusive); read from remote; ...
	OwtGet             // GET (with upgrading read-lock in the local-write path)
	OwtGetPrefetchLock // (used for maximum parallelism when prefetching)
	//
	// None of the above
	//
	OwtNone
)

func (*OWT) FromS

func (owt *OWT) FromS(s string)

func (OWT) String

func (owt OWT) String() (s string)

func (OWT) ToS

func (owt OWT) ToS() (s string)

type ObjAttrs

type ObjAttrs struct {
	Cksum    *cos.Cksum `json:"checksum,omitempty"`  // object checksum (cloned)
	CustomMD cos.StrKVs `json:"custom-md,omitempty"` // custom metadata: ETag, MD5, CRC, user-defined ...
	Ver      string     `json:"version,omitempty"`   // object version
	Atime    int64      `json:"atime,omitempty"`     // access time (nanoseconds since UNIX epoch)
	Size     int64      `json:"size,omitempty"`      // object size (bytes)
}

see also apc.HdrObjAtime et al. @ api/apc/const.go (and note that naming must be consistent)

func (*ObjAttrs) AtimeUnix

func (oa *ObjAttrs) AtimeUnix() int64

func (*ObjAttrs) Checksum

func (oa *ObjAttrs) Checksum() *cos.Cksum

func (*ObjAttrs) CopyFrom

func (oa *ObjAttrs) CopyFrom(oah cos.OAH, skipCksum bool)

clone OAH => ObjAttrs (see also lom.CopyAttrs)

func (*ObjAttrs) DelCustomKeys

func (oa *ObjAttrs) DelCustomKeys(keys ...string)

func (*ObjAttrs) Equal

func (oa *ObjAttrs) Equal(rem cos.OAH) (eq bool)

local <=> remote equality in the context of cold-GET and download. This function decides whether we need to go ahead and re-read the object from its remote location.

Other than a "binary" size and version checks, rest logic goes as follows: objects are considered equal if they have a) the same version and at least one matching checksum, or b) the same remote "source" and at least one matching checksum, or c) two matching checksums. (See also note below.)

Note that mismatch in any given checksum type immediately renders inequality and return from the function.

func (*ObjAttrs) FromHeader

func (oa *ObjAttrs) FromHeader(hdr http.Header) (cksum *cos.Cksum)

NOTE: returning checksum separately for subsequent validation

func (*ObjAttrs) FromLsoEntry added in v1.3.22

func (oa *ObjAttrs) FromLsoEntry(e *LsoEntry)

func (*ObjAttrs) GetCustomKey

func (oa *ObjAttrs) GetCustomKey(key string) (val string, exists bool)

func (*ObjAttrs) GetCustomMD

func (oa *ObjAttrs) GetCustomMD() cos.StrKVs

func (*ObjAttrs) SetCksum

func (oa *ObjAttrs) SetCksum(ty, val string)

func (*ObjAttrs) SetCustomKey

func (oa *ObjAttrs) SetCustomKey(k, v string)

func (*ObjAttrs) SetCustomMD

func (oa *ObjAttrs) SetCustomMD(md cos.StrKVs)

func (*ObjAttrs) SetSize added in v1.3.16

func (oa *ObjAttrs) SetSize(size int64)

func (*ObjAttrs) SizeBytes

func (oa *ObjAttrs) SizeBytes(_ ...bool) int64

func (*ObjAttrs) String

func (oa *ObjAttrs) String() string

func (*ObjAttrs) Version

func (oa *ObjAttrs) Version(_ ...bool) string

type ObjectProps

type ObjectProps struct {
	Bck Bck `json:"bucket"`
	ObjAttrs
	Name     string `json:"name"`
	Location string `json:"location"` // see also `GetPropsLocation`
	Mirror   struct {
		Paths  []string `json:"paths,omitempty"`
		Copies int      `json:"copies,omitempty"`
	} `json:"mirror"`
	EC struct {
		Generation   int64 `json:"generation"`
		DataSlices   int   `json:"data"`
		ParitySlices int   `json:"parity"`
		IsECCopy     bool  `json:"replicated"`
	} `json:"ec"`
	Present bool `json:"present"`
}

object properties NOTE: embeds system `ObjAttrs` that in turn includes custom user-defined NOTE: compare with `apc.LsoMsg`

type ParseURIOpts

type ParseURIOpts struct {
	DefaultProvider string // If set the provider will be used as provider.
	IsQuery         bool   // Determines if the URI should be parsed as query.
}

type PeriodConf

type PeriodConf struct {
	StatsTime     cos.Duration `json:"stats_time"`      // collect and publish stats; other house-keeping
	RetrySyncTime cos.Duration `json:"retry_sync_time"` // metasync retry
	NotifTime     cos.Duration `json:"notif_time"`      // (IC notifications)
}

NOTE: StatsTime is a one important timer

func (*PeriodConf) Validate

func (c *PeriodConf) Validate() error

type PeriodConfToSet added in v1.3.21

type PeriodConfToSet struct {
	StatsTime     *cos.Duration `json:"stats_time,omitempty"`
	RetrySyncTime *cos.Duration `json:"retry_sync_time,omitempty"`
	NotifTime     *cos.Duration `json:"notif_time,omitempty"`
}

global configuration

type PropsValidator

type PropsValidator interface {
	ValidateAsProps(arg ...any) error
}

type ProxyConf

type ProxyConf struct {
	PrimaryURL   string `json:"primary_url"`
	OriginalURL  string `json:"original_url"`
	DiscoveryURL string `json:"discovery_url"`
	NonElectable bool   `json:"non_electable"`
}

global configuration

type ProxyConfToSet added in v1.3.21

type ProxyConfToSet struct {
	PrimaryURL   *string `json:"primary_url,omitempty"`
	OriginalURL  *string `json:"original_url,omitempty"`
	DiscoveryURL *string `json:"discovery_url,omitempty"`
	NonElectable *bool   `json:"non_electable,omitempty"`
}

global configuration

type QueryBcks

type QueryBcks Bck

func (*QueryBcks) AddToQuery

func (qbck *QueryBcks) AddToQuery(query url.Values)

func (QueryBcks) Contains

func (qbck QueryBcks) Contains(other *Bck) bool

func (QueryBcks) Equal

func (qbck QueryBcks) Equal(bck *Bck) bool

func (*QueryBcks) IsAIS

func (qbck *QueryBcks) IsAIS() bool

func (*QueryBcks) IsBucket

func (qbck *QueryBcks) IsBucket() bool

QueryBcks is a Bck that _can_ have an empty Name. (TODO: extend to support prefix and regex.)

func (*QueryBcks) IsCloud

func (qbck *QueryBcks) IsCloud() bool

func (*QueryBcks) IsEmpty

func (qbck *QueryBcks) IsEmpty() bool

func (*QueryBcks) IsHDFS

func (qbck *QueryBcks) IsHDFS() bool

func (*QueryBcks) IsHTTP

func (qbck *QueryBcks) IsHTTP() bool

func (*QueryBcks) IsRemoteAIS

func (qbck *QueryBcks) IsRemoteAIS() bool

func (*QueryBcks) NewQuery added in v1.3.21

func (qbck *QueryBcks) NewQuery() url.Values

func (QueryBcks) String

func (qbck QueryBcks) String() string

func (*QueryBcks) Validate

func (qbck *QueryBcks) Validate() (err error)

type RebalanceConf

type RebalanceConf struct {
	Compression   string       `json:"compression"`       // enum { CompressAlways, ... } in api/apc/compression.go
	DestRetryTime cos.Duration `json:"dest_retry_time"`   // max wait for ACKs & neighbors to complete
	SbundleMult   int          `json:"bundle_multiplier"` // stream-bundle multiplier: num streams to destination
	Enabled       bool         `json:"enabled"`           // true=auto-rebalance | manual rebalancing
}

global configuration

func (*RebalanceConf) String

func (c *RebalanceConf) String() string

func (*RebalanceConf) Validate

func (c *RebalanceConf) Validate() error

type RebalanceConfToSet added in v1.3.21

type RebalanceConfToSet struct {
	DestRetryTime *cos.Duration `json:"dest_retry_time,omitempty"`
	Compression   *string       `json:"compression,omitempty"`
	SbundleMult   *int          `json:"bundle_multiplier"`
	Enabled       *bool         `json:"enabled,omitempty"`
}

global configuration

type ResilverConf

type ResilverConf struct {
	Enabled bool `json:"enabled"` // true=auto-resilver | manual resilvering
}

global configuration

func (*ResilverConf) String

func (c *ResilverConf) String() string

func (*ResilverConf) Validate

func (*ResilverConf) Validate() error

type ResilverConfToSet added in v1.3.21

type ResilverConfToSet struct {
	Enabled *bool `json:"enabled,omitempty"`
}

global configuration

type RetryArgs

type RetryArgs struct {
	Call    func() (int, error)
	IsFatal func(error) bool

	Action string
	Caller string

	SoftErr uint // How many retires on ConnectionRefused or ConnectionReset error.
	HardErr uint // How many retries on any other error.
	Sleep   time.Duration

	Verbosity int  // Determine the verbosity level.
	BackOff   bool // If requests should be retried less and less often.
	IsClient  bool // true: client (e.g. dev tools, etc.)
}

type SpaceConf

type SpaceConf struct {
	// Storage Cleanup watermark: used capacity (%) that triggers cleanup
	// (deleted objects and buckets, extra copies, etc.)
	CleanupWM int64 `json:"cleanupwm"`

	// LowWM: used capacity low-watermark (% of total local storage capacity)
	LowWM int64 `json:"lowwm"`

	// HighWM: used capacity high-watermark (% of total local storage capacity)
	// - LRU starts evicting objects when the currently used capacity (used-cap) gets above HighWM
	// - and keeps evicting objects until the used-cap gets below LowWM
	// - while self-throttling itself in accordance with target utilization
	HighWM int64 `json:"highwm"`

	// Out-of-Space: if exceeded, the target starts failing new PUTs and keeps
	// failing them until its local used-cap gets back below HighWM (see above)
	OOS int64 `json:"out_of_space"`
}

global configuration

func (*SpaceConf) String

func (c *SpaceConf) String() string

func (*SpaceConf) Validate

func (c *SpaceConf) Validate() (err error)

func (*SpaceConf) ValidateAsProps

func (c *SpaceConf) ValidateAsProps(...any) error

type SpaceConfToSet added in v1.3.21

type SpaceConfToSet struct {
	CleanupWM *int64 `json:"cleanupwm,omitempty"`
	LowWM     *int64 `json:"lowwm,omitempty"`
	HighWM    *int64 `json:"highwm,omitempty"`
	OOS       *int64 `json:"out_of_space,omitempty"`
}

global configuration

type TCBConf

type TCBConf struct {
	Compression string `json:"compression"`       // enum { CompressAlways, ... } in api/apc/compression.go
	SbundleMult int    `json:"bundle_multiplier"` // stream-bundle multiplier: num streams to destination
}

global configuration

func (*TCBConf) Validate

func (c *TCBConf) Validate() error

type TCBConfToSet added in v1.3.21

type TCBConfToSet struct {
	Compression *string `json:"compression,omitempty"`
	SbundleMult *int    `json:"bundle_multiplier,omitempty"`
}

global configuration

type TCObjsMsg

type TCObjsMsg struct {
	ToBck Bck `json:"tobck"`
	apc.TCObjsMsg
}

Multi-object copy & transform (see also: TCBMsg)

type TLSArgs added in v1.3.21

type TLSArgs struct {
	ClientCA    string
	Certificate string
	Key         string
	SkipVerify  bool
}

type TestFSPConf

type TestFSPConf struct {
	Root     string `json:"root"`
	Count    int    `json:"count"`
	Instance int    `json:"instance"`
}

local config

func (*TestFSPConf) Validate

func (c *TestFSPConf) Validate(contextConfig *Config) (err error)

validate root and (NOTE: testing only) generate and fill-in counted FSP.Paths

func (*TestFSPConf) ValidateMpath

func (c *TestFSPConf) ValidateMpath(p string) (err error)

type TimeoutConf

type TimeoutConf struct {
	CplaneOperation cos.Duration `json:"cplane_operation"` // read-mostly via global cmn.Rom.CplaneOperation
	MaxKeepalive    cos.Duration `json:"max_keepalive"`    // ditto, cmn.Rom.MaxKeepalive - see below
	MaxHostBusy     cos.Duration `json:"max_host_busy"`
	Startup         cos.Duration `json:"startup_time"`
	JoinAtStartup   cos.Duration `json:"join_startup_time"` // (join cluster at startup) timeout
	SendFile        cos.Duration `json:"send_file_time"`
}

maximum intra-cluster latencies (in the increasing order)

func (*TimeoutConf) Validate

func (c *TimeoutConf) Validate() error

type TimeoutConfToSet added in v1.3.21

type TimeoutConfToSet struct {
	CplaneOperation *cos.Duration `json:"cplane_operation,omitempty"`
	MaxKeepalive    *cos.Duration `json:"max_keepalive,omitempty"`
	MaxHostBusy     *cos.Duration `json:"max_host_busy,omitempty"`
	Startup         *cos.Duration `json:"startup_time,omitempty"`
	JoinAtStartup   *cos.Duration `json:"join_startup_time,omitempty"`
	SendFile        *cos.Duration `json:"send_file_time,omitempty"`
}

global configuration

type TransportArgs

type TransportArgs struct {
	DialTimeout      time.Duration
	Timeout          time.Duration
	IdleConnTimeout  time.Duration
	IdleConnsPerHost int
	MaxIdleConns     int
	SndRcvBufSize    int
	WriteBufferSize  int
	ReadBufferSize   int
	UseHTTPProxyEnv  bool
}

assorted http(s) client options

func (*TransportArgs) ConnControl

func (args *TransportArgs) ConnControl(_ syscall.RawConn) (cntl func(fd uintptr))

type TransportConf

type TransportConf struct {
	MaxHeaderSize int `json:"max_header"`   // max transport header buffer (default=4K)
	Burst         int `json:"burst_buffer"` // num sends with no back pressure; see also AIS_STREAM_BURST_NUM
	// two no-new-transmissions durations:
	// * IdleTeardown: sender terminates the connection (to reestablish it upon the very first/next PDU)
	// * QuiesceTime:  safe to terminate or transition to the next (in re: rebalance) stage
	IdleTeardown cos.Duration `json:"idle_teardown"`
	QuiesceTime  cos.Duration `json:"quiescent"`
	// lz4
	// max uncompressed block size, one of [64K, 256K(*), 1M, 4M]
	// fastcompression.blogspot.com/2013/04/lz4-streaming-format-final.html
	LZ4BlockMaxSize  cos.SizeIEC `json:"lz4_block"`
	LZ4FrameChecksum bool        `json:"lz4_frame_checksum"`
}

global configuration

func (*TransportConf) Validate

func (c *TransportConf) Validate() (err error)

NOTE: uncompressed block sizes - the enum currently supported by the github.com/pierrec/lz4

type TransportConfToSet added in v1.3.21

type TransportConfToSet struct {
	MaxHeaderSize    *int          `json:"max_header,omitempty" list:"readonly"`
	Burst            *int          `json:"burst_buffer,omitempty" list:"readonly"`
	IdleTeardown     *cos.Duration `json:"idle_teardown,omitempty"`
	QuiesceTime      *cos.Duration `json:"quiescent,omitempty"`
	LZ4BlockMaxSize  *cos.SizeIEC  `json:"lz4_block,omitempty"`
	LZ4FrameChecksum *bool         `json:"lz4_frame_checksum,omitempty"`
}

global configuration

type Validator

type Validator interface {
	Validate() error
}

type VersionConf

type VersionConf struct {
	// Determines if versioning is enabled
	Enabled bool `json:"enabled"`

	// Validate remote version and, possibly, update in-cluster ("cached") copy.
	// Scenarios include (but are not limited to):
	// - warm GET
	// - prefetch bucket (*)
	// - prefetch multiple objects (see api/multiobj.go)
	// - copy bucket
	// Applies to Cloud and remote AIS buckets - generally, buckets that have remote backends
	// that in turn provide some form of object versioning.
	// (*) Xactions (ie., jobs) that read-access multiple objects (e.g., prefetch, copy-bucket)
	// may support operation-scope option to synchronize remote content (to aistore) - the option
	// not requiring changing bucket configuration.
	// See also:
	// - apc.QparamLatestVer, apc.PrefetchMsg, apc.CopyBckMsg
	ValidateWarmGet bool `json:"validate_warm_get"`

	// A stronger variant of the above that in addition entails:
	// - deleting in-cluster object if its remote ("cached") counterpart does not exist
	// See also: apc.QparamSync, apc.CopyBckMsg
	Sync bool `json:"synchronize"`
}

global configuration

func (*VersionConf) String

func (c *VersionConf) String() string

func (*VersionConf) Validate

func (c *VersionConf) Validate() error

type VersionConfToSet added in v1.3.21

type VersionConfToSet struct {
	Enabled         *bool `json:"enabled,omitempty"`
	ValidateWarmGet *bool `json:"validate_warm_get,omitempty"`
	Sync            *bool `json:"synchronize,omitempty"`
}

global configuration

type WritePolicyConf

type WritePolicyConf struct {
	Data apc.WritePolicy `json:"data"`
	MD   apc.WritePolicy `json:"md"`
}

global configuration

func (*WritePolicyConf) Validate

func (c *WritePolicyConf) Validate() (err error)

func (*WritePolicyConf) ValidateAsProps

func (c *WritePolicyConf) ValidateAsProps(...any) error

type WritePolicyConfToSet added in v1.3.21

type WritePolicyConfToSet struct {
	Data *apc.WritePolicy `json:"data,omitempty" list:"readonly"` // NOTE: NIY
	MD   *apc.WritePolicy `json:"md,omitempty"`
}

global configuration

Directories

Path Synopsis
Package archive: write, read, copy, append, list primitives across all supported formats
Package archive: write, read, copy, append, list primitives across all supported formats
Package atomic provides simple wrappers around numerics to enforce atomic access.
Package atomic provides simple wrappers around numerics to enforce atomic access.
Package cos provides common low-level types and utilities for all aistore projects.
Package cos provides common low-level types and utilities for all aistore projects.
Package provides debug utilities
Package provides debug utilities
Package feat: global runtime-configurable feature flags
Package feat: global runtime-configurable feature flags
Package fname contains filename constants and common system directories
Package fname contains filename constants and common system directories
Package jsp (JSON persistence) provides utilities to store and load arbitrary JSON-encoded structures with optional checksumming and compression.
Package jsp (JSON persistence) provides utilities to store and load arbitrary JSON-encoded structures with optional checksumming and compression.
Package k8s: initialization, client, and misc.
Package k8s: initialization, client, and misc.
Package kvdb provides a local key/value database server for AIS.
Package kvdb provides a local key/value database server for AIS.
Package mono provides low-level monotonic time
Package mono provides low-level monotonic time
Package nlog - aistore logger, provides buffering, timestamping, writing, and flushing/syncing/rotating
Package nlog - aistore logger, provides buffering, timestamping, writing, and flushing/syncing/rotating
Package prob implements fully features dynamic probabilistic filter.
Package prob implements fully features dynamic probabilistic filter.
Package xoshiro256 implements the xoshiro256** RNG no-copyright
Package xoshiro256 implements the xoshiro256** RNG no-copyright

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL