cockroach: github.com/cockroachdb/cockroach/pkg/ccl/storageccl Index | Files | Directories

package storageccl

import "github.com/cockroachdb/cockroach/pkg/ccl/storageccl"

Index

Package Files

encryption.go export.go import.go key_rewriter.go revision_reader.go writebatch.go

Variables

var ExportRequestMaxAllowedFileSizeOverage = settings.RegisterByteSizeSetting(
    "kv.bulk_sst.max_allowed_overage",
    "if positive, allowed size in excess of target size for SSTs from export requests",
    64<<20,
)

ExportRequestMaxAllowedFileSizeOverage controls the maximum size in excess of the target file size which an exported SST may be. If this value is positive and an SST would exceed this size (due to large rows or large numbers of versions), then the export will fail.

var ExportRequestTargetFileSize = settings.RegisterByteSizeSetting(
    "kv.bulk_sst.target_size",
    "target size for SSTs emitted from export requests",
    64<<20,
)

ExportRequestTargetFileSize controls the target file size for SSTs created during backups.

func AppearsEncrypted Uses

func AppearsEncrypted(text []byte) bool

AppearsEncrypted checks if passed bytes begin with an encryption preamble.

func DecryptFile Uses

func DecryptFile(ciphertext, key []byte) ([]byte, error)

DecryptFile decrypts a file encrypted by EncryptFile, using the supplied key and reading the IV from a prefix of the file. See comments on EncryptFile for intended usage, and see DecryptFile

func EncryptFile Uses

func EncryptFile(plaintext, key []byte) ([]byte, error)

EncryptFile encrypts a file with the supplied key and a randomly chosen IV which is prepended in a header on the returned ciphertext. It is intended for use on collections of separate data files that are all encrypted/decrypted with the same key, such as BACKUP data files, and notably does _not_ include information for key derivation as that is _not_ done for individual files in such cases. See EncryptFileStoringSalt for a function that produces a ciphertext that also includes the salt and thus supports decrypting with only the passphrase (at the cost in including key derivation).

func GenerateKey Uses

func GenerateKey(passphrase, salt []byte) []byte

GenerateKey generates a key for the supplied passphrase and salt.

func GenerateSalt Uses

func GenerateSalt() ([]byte, error)

GenerateSalt generates a 16 byte random salt.

func ImportBufferConfigSizes Uses

func ImportBufferConfigSizes(st *cluster.Settings, isPKAdder bool) (int64, func() int64, int64)

ImportBufferConfigSizes determines the minimum, maximum and step size for the BulkAdder buffer used in import.

func MaxImportBatchSize Uses

func MaxImportBatchSize(st *cluster.Settings) int64

MaxImportBatchSize determines the maximum size of the payload in an AddSSTable request. It uses the ImportBatchSize setting directly unless the specified value would exceed the maximum Raft command size, in which case it returns the maximum batch size that will fit within a Raft command.

func SHA512ChecksumData Uses

func SHA512ChecksumData(data []byte) ([]byte, error)

SHA512ChecksumData returns the SHA512 checksum of data.

type KeyRewriter Uses

type KeyRewriter struct {
    // contains filtered or unexported fields
}

KeyRewriter rewrites old table IDs to new table IDs. It is able to descend into interleaved keys, and is able to function on partial keys for spans and splits.

func MakeKeyRewriter Uses

func MakeKeyRewriter(descs map[descpb.ID]*sqlbase.ImmutableTableDescriptor) (*KeyRewriter, error)

MakeKeyRewriter makes a KeyRewriter from a map of descs keyed by original ID.

func MakeKeyRewriterFromRekeys Uses

func MakeKeyRewriterFromRekeys(rekeys []roachpb.ImportRequest_TableRekey) (*KeyRewriter, error)

MakeKeyRewriterFromRekeys makes a KeyRewriter from Rekey protos.

func (*KeyRewriter) RewriteKey Uses

func (kr *KeyRewriter) RewriteKey(key []byte, isFromSpan bool) ([]byte, bool, error)

RewriteKey modifies key (possibly in place), changing all table IDs to their new value, including any interleaved table children and prefix ends. This function works by inspecting the key for table and index IDs, then uses the corresponding table and index descriptors to determine if interleaved data is present and if it is, to find the next prefix of an interleaved child, then calls itself recursively until all interleaved children have been rekeyed. If it encounters a table ID for which it does not have a configured rewrite, it returns the prefix of the key that was rewritten key. The returned boolean is true if and only if all of the table IDs found in the key were rewritten. If isFromSpan is true, failures in value decoding are assumed to be due to valid span manipulations, like PrefixEnd or Next having altered the trailing byte(s) to corrupt the value encoding -- in such a case we will not be able to decode the value (to determine how much further to scan for table IDs) but we can assume that since these manipulations are only done to the trailing byte that we're likely at the end anyway and do not need to search for any further table IDs to replace.

type VersionedValues Uses

type VersionedValues struct {
    Key    roachpb.Key
    Values []roachpb.Value
}

VersionedValues is similar to roachpb.KeyValue except instead of just the value at one time, it contains all the retrieved revisions of the value for the key, with the value timestamps set accordingly.

func GetAllRevisions Uses

func GetAllRevisions(
    ctx context.Context, db *kv.DB, startKey, endKey roachpb.Key, startTime, endTime hlc.Timestamp,
) ([]VersionedValues, error)

GetAllRevisions scans all keys between startKey and endKey getting all revisions between startTime and endTime. TODO(dt): if/when client gets a ScanRevisionsRequest or similar, use that.

Directories

PathSynopsis
engineccl
engineccl/enginepbccl

Package storageccl imports 36 packages (graph) and is imported by 114 packages. Updated 2020-08-09. Refresh now. Tools for package owners.