inventory

package
v0.21.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 23, 2024 License: MIT Imports: 31 Imported by: 0

Documentation

Overview

Package inventory provides the needed pieces to correctly create an Inventory of a directory

Index

Constants

View Source 
const (
	// NullFormat is the unset value for this type
	NullFormat = iota
	// YAMLFormat is for yaml
	YAMLFormat
	// JSONFormat is for yaml
	JSONFormat
)
View Source 
const DefaultSuitcaseFormat string = "tar.zst"

DefaultSuitcaseFormat is just the default format we're going to use for a suitcase. Hopefully this fits for most use cases, but can always be overridden

Variables

This section is empty.

Functions

func ArchiveTOC added in v0.18.0 

func ArchiveTOC(fn string) ([]string, error)

ArchiveTOC is a v4 TOC generator for archiver

func BindCobra added in v0.12.0 

func BindCobra(cmd *cobra.Command)

BindCobra binds the needed inventory bits to a cobra.Command

func FormatCompletion added in v0.11.0 

func FormatCompletion(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective)

FormatCompletion returns shell completion

func GetMetadataWithFiles 

func GetMetadataWithFiles(files []string) (map[string]string, error)

GetMetadataWithFiles returns the metadata for a set of files

func GetMetadataWithGlob 

func GetMetadataWithGlob(fpg string) (map[string]string, error)

GetMetadataWithGlob Given a file path with a glob, return metadata. The metadata is a map of filename to data

func HashCompletion added in v0.12.0 

func HashCompletion(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective)

HashCompletion returns shell completion

func NewInventoryCmd added in v0.18.0 

func NewInventoryCmd() *cobra.Command

NewInventoryCmd is a shortcut for an inventory command

func WithArchiveTOC added in v0.13.0 

func WithArchiveTOC() func(*Options)

WithArchiveTOC enables table of contents in the archive file inventory. This only checks files with a known archive extension.

func WithArchiveTOCDeep added in v0.14.4 

func WithArchiveTOCDeep() func(*Options)

WithArchiveTOCDeep enables table of contents in the archive file inventory. This checks every file, regardless of extension.

func WithCobra added in v0.12.0 

func WithCobra(cmd *cobra.Command, args []string) func(*Options)

WithCobra applies options using a cobra Command and args

func WithDirectories added in v0.12.0 

func WithDirectories(d []string) func(*Options)

WithDirectories sets the top level directories to be suitcased up 

func WithFollowSymlinks() func(*Options)

WithFollowSymlinks sets the FollowSymlinks option to true

func WithHashAlgorithms added in v0.12.0 

func WithHashAlgorithms(a HashAlgorithm) func(*Options)

WithHashAlgorithms sets the hashing algorithms to use for signatures

func WithIgnoreGlobs added in v0.12.0 

func WithIgnoreGlobs(g []string) func(*Options)

WithIgnoreGlobs sets the IgnoreGlobs strings

func WithInventoryFormat added in v0.12.0 

func WithInventoryFormat(f string) func(*Options)

WithInventoryFormat sets the format for the suitcases that will be generated

func WithLimitFileCount added in v0.12.0 

func WithLimitFileCount(c int) func(*Options)

WithLimitFileCount sets the number of files to process before stopping. 0 means process them all

func WithMaxSuitcaseSize added in v0.12.0 

func WithMaxSuitcaseSize(s int64) func(*Options)

WithMaxSuitcaseSize sets the maximum size for any of the generated suitcases

func WithPrefix added in v0.12.0 

func WithPrefix(p string) func(*Options)

WithPrefix sets the prefix for an inventory

func WithSuitcaseFormat added in v0.12.0 

func WithSuitcaseFormat(f string) func(*Options)

WithSuitcaseFormat sets the format for the suitcases that will be generated

func WithUser added in v0.12.0 

func WithUser(u string) func(*Options)

WithUser sets the user for an inventory option

func WithViper added in v0.12.0 

func WithViper(v *viper.Viper) func(*Options)

WithViper applies options from a Viper instance for options

func WithWizardForm added in v0.20.0 

func WithWizardForm(f WizardForm) func(*Options)

WithWizardForm sets up data from a wizard form

Types

type Analysis added in v0.14.0 

type Analysis struct {
	LargestFileSize   int64
	LargestFileSizeHR string
	FileCount         uint
	AverageFileSize   int64
	AverageFileSizeHR string
	TotalFileSize     int64
	TotalFileSizeHR   string
}

Analysis is some useful information about a given inventory

type CaseSet 

type CaseSet map[int]int64

CaseSet is just a holder for case sizes

func NewCaseSet 

func NewCaseSet(maxSize int64) CaseSet

NewCaseSet returns a new set of suitcase params This is used to keep track of sizing

type Collection added in v0.14.0 

type Collection map[string]Inventory

Collection is map of inventory paths to Inventory objects

func CollectionWithDirs added in v0.14.0 

func CollectionWithDirs(d []string) (*Collection, error)

CollectionWithDirs returns a Collection of inventories using a list of directories

type EJSONer 

type EJSONer struct{}

EJSONer is the easy json operator

func (EJSONer) Read 

func (r EJSONer) Read(b []byte) (*Inventory, error)

Read reads the bytes and returns an inventory

func (*EJSONer) Write 

func (r *EJSONer) Write(w io.Writer, i *Inventory) error

Write writes out an inventory file

type File 

type File struct {
	Path          string   `yaml:"path" json:"path"`
	Destination   string   `yaml:"destination" json:"destination"`
	Name          string   `yaml:"name" json:"name"`
	Size          int64    `yaml:"size" json:"size"`
	ArchiveTOC    []string `yaml:"archive_toc,omitempty" json:"archive_toc,omitempty"`
	SuitcaseIndex int      `yaml:"suitcase_index,omitempty" json:"suitcase_index,omitempty"`
	SuitcaseName  string   `yaml:"suitcase_name,omitempty" json:"suitcase_name,omitempty"`
}

File is a file item inside an inventory

func (File) MarshalEasyJSON 

func (v File) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (File) MarshalJSON 

func (v File) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*File) UnmarshalEasyJSON 

func (v *File) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*File) UnmarshalJSON 

func (v *File) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type FileBucket 

type FileBucket struct {
	Free int64
}

FileBucket describes what a filebucket state is

func (FileBucket) MarshalEasyJSON 

func (v FileBucket) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (FileBucket) MarshalJSON 

func (v FileBucket) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*FileBucket) UnmarshalEasyJSON 

func (v *FileBucket) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*FileBucket) UnmarshalJSON 

func (v *FileBucket) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type Format added in v0.11.0 

type Format int

Format is the format the inventory will use, such as yaml, json, etc

func (*Format) MarshalJSON added in v0.11.0 

func (f *Format) MarshalJSON() ([]byte, error)

MarshalJSON ensures that json conversions use the string value here, not the int value

func (*Format) Set added in v0.11.0 

func (f *Format) Set(v string) error

Set helps fulfill the pflag.Value interface

func (Format) String added in v0.11.0 

func (f Format) String() string

func (Format) Type added in v0.11.0 

func (f Format) Type() string

Type satisfies part of the pflags.Value interface

type HashAlgorithm added in v0.12.0 

type HashAlgorithm int

HashAlgorithm is the hashing algorithm used for calculating file signatures 

const (
	// NullHash represents no hashing
	NullHash HashAlgorithm = iota
	// MD5Hash uses and md5 checksum
	MD5Hash
	// SHA1Hash is the sha-1 version of a signature
	SHA1Hash
	// SHA256Hash is the more secure sha-256 version of a signature
	SHA256Hash
	// SHA512Hash is most secure, but super slow, probably not useful here
	SHA512Hash
)

func (*HashAlgorithm) MarshalJSON added in v0.12.0 

func (h *HashAlgorithm) MarshalJSON() ([]byte, error)

MarshalJSON ensures that json conversions use the string value here, not the int value

func (*HashAlgorithm) Set added in v0.12.0 

func (h *HashAlgorithm) Set(v string) error

Set helps fulfill the pflag.Value interface

func (HashAlgorithm) String added in v0.12.0 

func (h HashAlgorithm) String() string

String satisfies the pflags interface

func (HashAlgorithm) Type added in v0.12.0 

func (h HashAlgorithm) Type() string

Type satisfies part of the pflags.Value interface

type IndexSummary 

type IndexSummary struct {
	Count     uint   `yaml:"count"`
	Size      int64  `yaml:"size"`
	HumanSize string `yaml:"human_size"`
}

IndexSummary will give an overall summary to a set of suitcases

func (IndexSummary) MarshalEasyJSON 

func (v IndexSummary) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (IndexSummary) MarshalJSON 

func (v IndexSummary) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*IndexSummary) UnmarshalEasyJSON 

func (v *IndexSummary) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*IndexSummary) UnmarshalJSON 

func (v *IndexSummary) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type Inventory added in v0.14.5 

type Inventory struct {
	Files            []*File               `yaml:"files" json:"files"`
	Options          *Options              `yaml:"options" json:"options"`
	TotalIndexes     int                   `yaml:"total_indexes" json:"total_indexes"`
	IndexSummaries   map[int]*IndexSummary `yaml:"index_summaries" json:"index_summaries"`
	InternalMetadata map[string]string     `yaml:"internal_metadata" json:"internal_metadata"`
	ExternalMetadata map[string]string     `yaml:"external_metadata" json:"external_metadata"`
}

Inventory is the inventory of a set of suitcases

func NewDirectoryInventory 

func NewDirectoryInventory(opts *Options) (*Inventory, error)

NewDirectoryInventory creates a new DirectoryInventory using options

func NewInventoryWithFilename 

func NewInventoryWithFilename(s string) (*Inventory, error)

NewInventoryWithFilename returns a new DirectoryInventory from an inventory File

func (Inventory) Analyze added in v0.14.5 

func (di Inventory) Analyze() Analysis

Analyze examines an inventory and returns an Analysis object

func (*Inventory) IndexWithSize added in v0.14.5 

func (di *Inventory) IndexWithSize(maxSize int64) error

IndexWithSize Loops through inventory and assign suitcase indexes based on a given max size

func (Inventory) JSONString added in v0.18.0 

func (di Inventory) JSONString() (string, error)

JSONString returns the inventory in JSON and an optional error

func (Inventory) MarshalEasyJSON added in v0.14.5 

func (v Inventory) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (Inventory) MarshalJSON added in v0.14.5 

func (v Inventory) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (Inventory) MustJSONString added in v0.18.0 

func (di Inventory) MustJSONString() string

MustJSONString returns the json representation as a string or panic

func (Inventory) Search added in v0.14.5 

func (di Inventory) Search(p string) SearchResults

Search iterates through files inside of an inventory and returns a set of results

func (*Inventory) SuitcaseNameWithIndex added in v0.14.5 

func (di *Inventory) SuitcaseNameWithIndex(i int) string

SuitcaseNameWithIndex gives what the name of a suitcase file will be, given the index number

func (Inventory) SuitcaseNames added in v0.14.5 

func (di Inventory) SuitcaseNames() []string

SuitcaseNames returns a list of suitcase names as strings

func (Inventory) SummaryLog added in v0.14.5 

func (di Inventory) SummaryLog()

SummaryLog logs out a summary of the suitcase data

func (Inventory) UniqueSuitcaseNames added in v0.14.5 

func (di Inventory) UniqueSuitcaseNames() []string

UniqueSuitcaseNames returns a list of suitcase names as strings

func (*Inventory) UnmarshalEasyJSON added in v0.14.5 

func (v *Inventory) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*Inventory) UnmarshalJSON added in v0.14.5 

func (v *Inventory) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

func (Inventory) ValidateAccess added in v0.21.0 

func (di Inventory) ValidateAccess() error

ValidateAccess ensures that we have access to all files in a given inventory

type Inventoryer 

type Inventoryer interface {
	Write(io.Writer, *Inventory) error
	Read([]byte) (*Inventory, error)
}

Inventoryer is an interface to define what an Inventory Operator does

func NewInventoryerWithFilename 

func NewInventoryerWithFilename(filename string) (Inventoryer, error)

NewInventoryerWithFilename creates a new inventoryer with a filename

type Options added in v0.12.0 

type Options struct {
	User                  string                   `yaml:"user" json:"user"`
	Prefix                string                   `yaml:"prefix" json:"prefix"`
	Directories           []string                 `yaml:"top_level_directories" json:"top_level_directories"`
	SizeConsideredLarge   int64                    `yaml:"size_considered_large" json:"size_considered_large"`
	MaxSuitcaseSize       int64                    `yaml:"max_suitcase_size" json:"max_suitcase_size"`
	InternalMetadataGlob  string                   `yaml:"internal_metadata_glob,omitempty" json:"internal_metadata_glob,omitempty"`
	IgnoreGlobs           []string                 `yaml:"ignore_globs,omitempty" json:"ignore_globs,omitempty"`
	ExternalMetadataFiles []string                 `yaml:"external_metadata_files,omitempty" json:"external_metadata_files,omitempty"`
	EncryptInner          bool                     `yaml:"encrypt_inner" json:"encrypt_inner"`
	HashInner             bool                     `yaml:"hash_inner" json:"hash_inner"`
	LimitFileCount        int                      `yaml:"limit_file_count" json:"limit_file_count"`
	SuitcaseFormat        string                   `yaml:"suitcase_format" json:"suitcase_format"`
	InventoryFormat       string                   `yaml:"inventory_format" json:"inventory_format"`
	FollowSymlinks        bool                     `yaml:"follow_symlinks" json:"follow_symlinks"`
	HashAlgorithm         HashAlgorithm            `yaml:"hash_algorithm" json:"hash_algorithm"`
	IncludeArchiveTOC     bool                     `yaml:"include_archive_toc" json:"include_archive_toc"`
	IncludeArchiveTOCDeep bool                     `yaml:"include_archive_toc_deep" json:"include_archive_toc_deep"`
	TransportPlugin       transporters.Transporter `yaml:"transport_plugin" json:"transport_plugin"`
}

Options are the options used to create a DirectoryInventory

func NewOptions added in v0.11.5 

func NewOptions(options ...func(*Options)) *Options

NewOptions uses functional options to generatea DirectoryInventoryOptions object

func (*Options) AbsoluteDirectories added in v0.12.0 

func (o *Options) AbsoluteDirectories() error

AbsoluteDirectories converts the Directories entries to absolute paths

func (Options) MarshalEasyJSON added in v0.12.0 

func (v Options) MarshalEasyJSON(w *jwriter.Writer)

MarshalEasyJSON supports easyjson.Marshaler interface

func (Options) MarshalJSON added in v0.12.0 

func (v Options) MarshalJSON() ([]byte, error)

MarshalJSON supports json.Marshaler interface

func (*Options) UnmarshalEasyJSON added in v0.12.0 

func (v *Options) UnmarshalEasyJSON(l *jlexer.Lexer)

UnmarshalEasyJSON supports easyjson.Unmarshaler interface

func (*Options) UnmarshalJSON added in v0.12.0 

func (v *Options) UnmarshalJSON(data []byte) error

UnmarshalJSON supports json.Unmarshaler interface

type SearchDirMatch added in v0.13.0 

type SearchDirMatch struct {
	Directory   string
	TotalSize   uint64
	TotalSizeHR string
	Suitcases   []string
}

SearchDirMatch is a single directory match

type SearchDirMatches added in v0.13.0 

type SearchDirMatches []SearchDirMatch

SearchDirMatches is a recursive listing of what is contained in a directory

type SearchFileMatches added in v0.13.0 

type SearchFileMatches []File

SearchFileMatches is a listing of files that match a given search

type SearchResults added in v0.13.0 

type SearchResults struct {
	Files       SearchFileMatches
	Directories SearchDirMatches
}

SearchResults is all the stuff that a given search gets back

type VAMLer 

type VAMLer struct{}

VAMLer is the Victor YAML for operator

func (VAMLer) Read 

func (r VAMLer) Read(b []byte) (*Inventory, error)

Read will ready byte in to an inventory

func (*VAMLer) Write 

func (r *VAMLer) Write(w io.Writer, i *Inventory) error

Write will write the inventory out to an io.Writer

type WizardForm added in v0.20.0 

type WizardForm struct {
	Destination      string
	Source           string
	TravelAgentToken string
	MaxSize          string
}

WizardForm are the little fields from our nice lil wizard 🧙

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.