core

package
v0.0.0-...-2386a5e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 6, 2015 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	GitRefTypeLocalBranch  = GitRefType(iota)
	GitRefTypeRemoteBranch = GitRefType(iota)
	GitRefTypeLocalTag     = GitRefType(iota)
	GitRefTypeRemoteTag    = GitRefType(iota)
	GitRefTypeHEAD         = GitRefType(iota) // current checkout
	GitRefTypeOther        = GitRefType(iota) // stash or unknown
)
View Source 
const ApproximateMetadataSize = 75
View Source 
const BUFSIZE = 131072
View Source 
const SHALen = 40
View Source 
const SHALineLen = len(SHAPrefix) + SHALen
View Source 
const SHALineMatchRegexStr = "^git-lob: ([0-9A-Fa-f]{40})$"
View Source 
const SHALineRegexStr = "^git-lob: [A-Fa-f0-9]{40}$"
View Source 
const SHAPrefix = "git-lob: "

Prefix which identifies file contents as a git-lob SHA entry Use this prefix rather than just the SHA in case by accident a file's content actually is a 40-char pattern

Variables

View Source 
var ChunkSize = int64(32 * 1024 * 1024)

Chunk size that we split stored data into so it's easier to resume uploads/downloads This used to be configurable, but it caused too many issues if different people had different settings in a shared repository This is only 'var' rather than 'const' to allow tests to modify

View Source 
var IsSHARegex *regexp.Regexp = regexp.MustCompile("^[0-9A-Fa-f]{8,40}$")

Functions

func ApplyLOBDelta 

func ApplyLOBDelta(basesha, targetsha string, delta io.Reader) error

Applies a diff to basesha and generates a LOB which should have targetsha (will be checked, error returned if disagrees)

func ApplyLOBDeltaInBaseDir 

func ApplyLOBDeltaInBaseDir(basedir, basesha, targetsha string, delta io.Reader) error

Applies a diff to basesha and generates a LOB, with a specified root storage, which should have targetsha (will be checked, error returned if disagrees)

func AutoFetch 

func AutoFetch(lobsha string, reportProgress bool) error

Auto-fetch a single LOB from the default locations If the required files are not found this won't cause an error

func CheckLOBFilesForSHA 

func CheckLOBFilesForSHA(sha, basedir string, checkHash bool) error

Check the integrity of the files for a given sha in the attached basedir If checkHash = true, reads all the data in the files and re-calculates the SHA for a deep validation of content (slower but complete) If checkHash = false, just checks the presence & size of all files (quick & most likely correct) Note that if basedir is the local root, will try to recover missing files from shared store

func CheckRemoteLOBFilesForSHA 

func CheckRemoteLOBFilesForSHA(sha string, provider providers.SyncProvider, remoteName string) error

Check with a remote provider for the presence of all data required for a given LOB Return nil if all data is there, NotFoundErr if not

func CheckRemoteLOBFilesForSHABasic 

func CheckRemoteLOBFilesForSHABasic(sha string, provider providers.SyncProvider, remoteName string) error

CheckRemoteLOBFilesForSHA on non-smart providers

func CheckRemoteLOBFilesForSHASmart 

func CheckRemoteLOBFilesForSHASmart(sha string, provider providers.SmartSyncProvider, remoteName string) error

CheckRemoteLOBFilesForSHA on mart providers

func Checkout 

func Checkout(pathspecs []string, dryRun bool, callback CheckoutCallback) error

Populate local placeholders with real content, if available. Do entire working copy unless limited to pathspecs

func CleanFilterWithReaderWriter 

func CleanFilterWithReaderWriter(in io.Reader, out io.Writer, filename string) int

func CleanupPushState 

func CleanupPushState(remoteName string)

Minimise the amount of state we retain on pushed state As we add SHAs that are pushed we can create redundant records because some SHAs are parents of others. This makes the subsequent retrieval of commits to push slower So remove SHAs that are ancestors of others and just keep the later SHAs that are pushed

func ConvertFileLOBSliceToMap 

func ConvertFileLOBSliceToMap(slice []*FileLOB) map[string]string

Convert a slice of FileLOBs to a map of lob sha to filename, eliminates duplicates 

func CreateHardLink(target, link string) error

Create a hard link to a file This link can be deleted like any other file afterwards

func DeleteLOB 

func DeleteLOB(sha string) error

Delete all files associated with a given LOB SHA from the local store

func DeleteLOBInBaseDir 

func DeleteLOBInBaseDir(sha, basedir string) error

Delete all files associated with a given LOB SHA from a specified root dir

func Fetch 

func Fetch(provider providers.SyncProvider, remoteName string, refspecs []*GitRefSpec, dryRun, force bool,
	callback util.ProgressCallback) error

Implementation of fetch

func FetchSingle 

func FetchSingle(lobsha string, provider providers.SyncProvider, remoteName string, force bool, callback util.ProgressCallback) error

Fetch the files required for a single LOB

func FindLatestAncestorWhereBinariesPushed 

func FindLatestAncestorWhereBinariesPushed(remoteName, ref string) (string, error)

Find the most recent ancestor of ref (or itself) at which we believe we've already pushed all binaries. Returns a blank string if none have been pushed.

func FormatGitDate 

func FormatGitDate(t time.Time) string

Format a date into Git format

func Fsck 

func Fsck(deep, shared, deleteBadFiles bool, shas []string, callback func(data *FsckCallbackData) (quit bool)) error

Validate local binary store deep = recaculate SHAs & check (takes longer) instead of just checking file size shared = check shared store instead of local store deleteBadFiles = delete files which are the wrong size or corrupted shas = specific list of binaries to check; if empty, checks entire store callback = for progress and file errors, return quit to abort (also skips deleting current item) The returned error will be nil if no files had any issues but the process will continue (with callbacks) even when missing/bad files are encountered until all have been checked

func GenerateLOBDelta 

func GenerateLOBDelta(basesha, targetsha string, out io.Writer) (int64, error)

Generates a diff between the contents of 2 LOBs Automatically copes with chunking, the diff is one file across the entire content Returns the size of the compressed delta

func GenerateLOBDeltaInBaseDir 

func GenerateLOBDeltaInBaseDir(basedir, basesha, targetsha string, out io.Writer) (int64, error)

Generates a diff between the contents of 2 LOBs, with a specified root storage Automatically copes with chunking, the diff is one file across the entire content Returns the size of the compressed delta

func GetGitAllLOBHistoryForFile 

func GetGitAllLOBHistoryForFile(filename, shatoskip string) ([]string, error)

Return a slice of LOB SHAs representing versions of filename, ordered by latest first history is from all heads not just checked out if shatoskip is supplied, this sha is excluded from the return if found

func GetGitAllLOBsToCheckoutAtCommit 

func GetGitAllLOBsToCheckoutAtCommit(commit string, includePaths, excludePaths []string) ([]string, error)

Get all the LOB SHAs that you would need to check out at a given commit (not changed in that commit)

func GetGitAllLOBsToCheckoutAtCommitAndRecent 

func GetGitAllLOBsToCheckoutAtCommitAndRecent(commit string, days int, includePaths,
	excludePaths []string) (lobs []string, earliestChangeCommit string, reterr error)

Get all the LOB SHAs that you would need to have available to check out a commit, and any other ancestor of it within a number of days of that commit date (not today's date) Note that if a LOB was modified to the same SHA more than once, duplicates may appear in the return They are not routinely eliminated for performance, so perform your own dupe removal if you need it as well as a list of LOBs, returns the commit SHA of the earliest change that was included in the scan. Since this is the first *change* included (which would be removing the previous SHA), the earliest LOB SHA included is from the *parent* of this commit.

func GetGitAllLOBsToCheckoutInRefSpec 

func GetGitAllLOBsToCheckoutInRefSpec(refspec *GitRefSpec, includePaths, excludePaths []string) ([]string, error)

Gets a list of LOB SHAs for all binary files that are needed when checking out any of the commits referred to by refspec. As opposed to GetGitCommitsReferencingLOBsInRange which only picks up changes to LOBs, this function returns the complete set of LOBs needed if you checked out a commit either at a single commit, or any in a range (if the refspec is a range; only .. range operator allowed) This means it will include any LOBs that were added in commits before the range, if they are still used, while GetGitCommitsReferencingLOBsInRange wouldn't mention those. Note that git ranges are start AND end inclusive in this case. Note that duplicate SHAs are not eliminated for efficiency, you must do it if you need it

func GetGitBestAncestor 

func GetGitBestAncestor(refs []string) (ancestor string, err error)

Returns the 'best' ancestor of all the passed in refs (as a SHA) If a ref is listed twice the 'best' ancestor will be itself

func GetGitCurrentBranch 

func GetGitCurrentBranch() string

Get the name of the current branch

func GetGitDefaultRemoteForPull 

func GetGitDefaultRemoteForPull() string

Gets the default fetch remote for the working dir Determined from tracking state of current branch if present, or defaults to origin.

func GetGitDefaultRemoteForPush 

func GetGitDefaultRemoteForPush() string

Gets the default push remote for the working dir Determined from branch.*.remote configuration for the current branch if present, or defaults to origin.

func GetGitLocalBranches 

func GetGitLocalBranches() ([]string, error)

Return a list of all local branches Also FYI caches the current branch while we're at it so it's zero-cost to call GetGitCurrentBranch after this

func GetGitPushDefaultBranches 

func GetGitPushDefaultBranches(remoteName string) []string

Return a list of branches to push by default, based on push.default and local/remote branches See push.default docs at https://www.kernel.org/pub/software/scm/git/docs/git-config.html

func GetGitRemoteBranches 

func GetGitRemoteBranches(remoteName string) ([]string, error)

Return a list of all remote branches for a given remote Note this doesn't retrieve mappings between local and remote branches, just a simple list

func GetGitRemotes 

func GetGitRemotes() ([]string, error)

Get a list of git remotes

func GetGitUpstreamBranch 

func GetGitUpstreamBranch(localbranch string) (remoteName, remoteBranch string)

Get the upstream branch for a given local branch, as defined in what 'git pull' would do by default returns the remote name and the remote branch separately for ease of use

func GetHardLinkCount 

func GetHardLinkCount(target string) (linkCount int, err error)

Get the number of hard links to a given file (min 1)

func GetLOBChunkPathInBaseDir 

func GetLOBChunkPathInBaseDir(basedir, sha string, chunkIdx int) string

Gets the absolute path to the chunk file for a LOB from a base dir

func GetLOBChunkRelativePath 

func GetLOBChunkRelativePath(sha string, chunkIdx int) string

Get a relative file name for a meta file (no dirs created as not rooted)

func GetLOBCompleteContent 

func GetLOBCompleteContent(sha string, out io.Writer) error

Retrieve the entire content for all chunks of a LOB and write to 'out'

func GetLOBCompleteContentInBaseDir 

func GetLOBCompleteContentInBaseDir(basedir, sha string, out io.Writer) error

Retrieve the entire content for all chunks of a LOB within a base root, and write to 'out'

func GetLOBFilesForSHA 

func GetLOBFilesForSHA(sha, basedir string, check bool, checkHash bool) (files []string, size int64, _err error)

Get the local/shared storage of a LOB with a given SHA Returns the list of files (relative to basedir) & checks for integrity if check = true If check = true and checkHash = true, reads all the data in the files and re-calculates the SHA for a deep validation of content If check = true and checkHash = false, just checks the presence & size of all files If there are any errors the returned list may not be correct In the rare case that a break has occurred between shared storage and the local hardlink, this method will re-link if the shared store has it

func GetLOBMetaPathInBaseDir 

func GetLOBMetaPathInBaseDir(basedir, sha string) string

Gets the absolute path to the meta file for a LOB from a base dir

func GetLOBMetaRelativePath 

func GetLOBMetaRelativePath(sha string) string

Get a relative file name for a meta file (no dirs created as not rooted)

func GetLocalLOBChunkPath 

func GetLocalLOBChunkPath(sha string, chunkIdx int) string

Gets the absolute path to the chunk file for a LOB in local store

func GetLocalLOBDir 

func GetLocalLOBDir(sha string) string

Gets the containing local folder for a given LOB SHA & creates if necessary LOBs are 'splayed' 2-levels deep based on first 6 chars of SHA (3 for each dir) We splay by 2 levels and by 3 each (4096 dirs) because we don't pack like git so need to ensure directory contents remain practical at high numbers of files

func GetLocalLOBMetaPath 

func GetLocalLOBMetaPath(sha string) string

Gets the absolute path to the meta file for a LOB in local store

func GetLocalLOBRoot 

func GetLocalLOBRoot() string

Gets the root directory for local LOB files & creates if necessary

func GetMissingLOBs 

func GetMissingLOBs(lobshas []string, checkHash bool) []string

Check the presence & integrity of the files for a given list of shas in this repo and return a list of those which failed the check If checkHash = true, reads all the data in the files and re-calculates the SHA for a deep validation of content (slower but complete) If checkHash = false, just checks the presence & size of all files (quick & most likely correct)

func GetPushedCommits 

func GetPushedCommits(remoteName string) []string

Get a list of commits that have been pushed for a remote remoteName can be "*" to return pushed list for all remotes combined

func GetSharedLOBChunkPath 

func GetSharedLOBChunkPath(sha string, chunkIdx int) string

Gets the absolute path to the chunk file for a LOB in local store

func GetSharedLOBDir 

func GetSharedLOBDir(sha string) string

Gets the containing shared folder for a given LOB SHA & creates if necessary LOBs are 'splayed' 2-levels deep based on first 6 chars of SHA (3 for each dir) We splay by 2 levels and by 3 each (4096 dirs) because we don't pack like git so need to ensure directory contents remain practical at high numbers of files

func GetSharedLOBRoot 

func GetSharedLOBRoot() string

Gets the root directory for shared LOB files & creates if necessary

func GitIsAncestor 

func GitIsAncestor(a, b string) (bool, error)

Returns whether commit a (sha or ref) is an ancestor of commit b (sha or ref)

func GitRefIsFullSHA 

func GitRefIsFullSHA(ref string) bool

Return whether a single git reference (not refspec, so no ranges) is a full SHA or not SHAs can be used directly for things like lob lookup but other refs have too be converted This version requires a full length SHA (40 characters)

func GitRefIsSHA 

func GitRefIsSHA(ref string) bool

Return whether a single git reference (not refspec, so no ranges) is a SHA or not SHAs can be used directly for things like lob lookup but other refs have too be converted This version accepts SHAs that are 8-40 characters in length, so accepts short SHAs

func GitRefOrSHAIsValid 

func GitRefOrSHAIsValid(refOrSHA string) bool

Returns whether a ref or SHA refers to a valid, existing commit or not by asking git to resolve it

func GitRefToFullSHA 

func GitRefToFullSHA(ref string) (string, error)

func GitRefreshIndexForFiles 

func GitRefreshIndexForFiles(files []string) error

Tell the index to refresh for files which we've modified outside of git commands This is necessary because git caches stat() info to provide a fast way to detect modifications for git-status and so can consider files modified when they're actually not when we've changed things that the filter would consider unmodified when called via git-diff. 'files' is a list of files with paths relative to the repo root

func HasPushedBinaryState 

func HasPushedBinaryState(remoteName string) bool

Do we have any pushed binary state recorded for a remote?

func InitSuccessfullyPushedCacheIfAppropriate 

func InitSuccessfullyPushedCacheIfAppropriate() bool

Initialise the 'pushed' markers for all recent commits, if we can be sure we can do it Most common case: just after clone Returns whether we met the requirements to do this

func IsGitRemote 

func IsGitRemote(remoteName string) bool

func IsIntegrityError 

func IsIntegrityError(err error) bool

Is an error an IntegrityError?

func IsLOBMissing 

func IsLOBMissing(sha string, checkHash bool) bool

Return whether a single LOB is missing

func IsLocalLOBStoreEmpty 

func IsLocalLOBStoreEmpty() bool

returns whether the local store has any binaries in it

func IsNotFoundError 

func IsNotFoundError(err error) bool

Is an error a NotFoundError?

func IsUsingSharedStorage 

func IsUsingSharedStorage() bool

func IsWrongSizeError 

func IsWrongSizeError(err error) bool

Is an error a NotFoundError?

func MarkAllBinariesPushed 

func MarkAllBinariesPushed(remoteName string) error

func MarkBinariesAsPushed 

func MarkBinariesAsPushed(remoteName, commitSHA, replaceCommitSHA string) error

Record that binaries have been pushed to a given remote at a commit replaceCommitSHA can be blank, but if provided will replace a previously inserted SHA If you use replaceCommitSHA, it MUST BE an ancestor of commitSHA

func Missing 

func Missing(checkout bool, paths []string, callback func(data *MissingCallbackData) (quit bool))

Check for placeholders

func NewIntegrityError 

func NewIntegrityError(shas []string) error

Create a new IntegrityError

func NewIntegrityErrorWithAdditionalMessage 

func NewIntegrityErrorWithAdditionalMessage(shas []string, msg string) error

func NewNotFoundError 

func NewNotFoundError(msg, path string) error

Create a new NotFound error

func NewWrongSizeError 

func NewWrongSizeError(msg, filename string) error

Create a new WrongSize error

func ParseGitDate 

func ParseGitDate(str string) (time.Time, error)

Parse a Git date formatted in ISO 8601 format (%ci/%ai)

func PruneOld 

func PruneOld(dryRun, safeMode bool, callback PruneCallback) ([]string, error)

Remove LOBs from the local store if they fall outside the range we would normally fetch for Returns a list of SHAs that were deleted (unless dryRun = true) Unreferenced binaries are also deleted by this

func PruneSharedStore 

func PruneSharedStore(dryRun bool, callback PruneCallback) ([]string, error)

Prune the shared store of all LOBs with only 1 hard link (itself) DeleteLOB will do this for individual LOBs we prune, but if the user manually deletes a repo then unreferenced shared LOBs may never be cleaned up callback is a basic function to let caller know something is happening

func PruneUnreferenced 

func PruneUnreferenced(dryRun bool, callback PruneCallback) ([]string, error)

Delete unreferenced binary files from local store For a file to be deleted it needs to not be referenced by any (reachable) commit Returns a list of SHAs that were deleted (unless dryRun = true)

func Push 

func Push(provider providers.SyncProvider, remoteName string, refspecs []*GitRefSpec, dryRun, force, recheck bool,
	callback util.ProgressCallback) error

func PushSingle 

func PushSingle(sha string, provider providers.SyncProvider, remoteName string, force bool,
	callback util.ProgressCallback) error

Push a single LOB to a remote

func ResetPushedBinaryState 

func ResetPushedBinaryState(remoteName string) error

Reset the cached information about which binaries we have cached for a given remote Warning: this will make the next push expensive while it recalculates

func SmudgeFilterWithReaderWriter 

func SmudgeFilterWithReaderWriter(in io.Reader, out io.Writer, filename string) int

func StoreLOBChunk 

func StoreLOBChunk(sha string, chunkNo int, fromChunkFile string, sz int64) error

Write the contents of fromFile to final storage with sha, checking the size If file already exists and is of the right size, will do nothing fromChunkFile will be moved into its final location or deleted if the data is already valid, so the file will not exist after this call (renamed to final location or deleted), unless error

func StoreLOBChunkInBaseDir 

func StoreLOBChunkInBaseDir(basedir, sha string, chunkNo int, fromChunkFile string, sz int64) error

Write the contents of fromFile to final storage with sha, checking the size, to a relative root If file already exists and is of the right size, will do nothing fromChunkFile will be moved into its final location or deleted if the data is already valid, so the file will not exist after this call (renamed to final location or deleted), unless error

func StoreLOBInfo 

func StoreLOBInfo(info *LOBInfo) error

Store the metadata for a given sha If it already exists and is of the right size, will do nothing

func StoreLOBInfoInBaseDir 

func StoreLOBInfoInBaseDir(basedir string, info *LOBInfo) error

Store the metadata for a given sha in a relative path If it already exists and is of the right size, will do nothing

func WalkGitAllLOBsToCheckoutAtCommit 

func WalkGitAllLOBsToCheckoutAtCommit(commit string, includePaths, excludePaths []string,
	callback func(filelob *FileLOB)) error

Utility function to walk through all the LOBs which are present if checked out at a specific commit

func WalkGitCommitLOBsToPush 

func WalkGitCommitLOBsToPush(remoteName, ref string, recheck bool, callback func(commitLOB *CommitLOBRef) (quit bool, err error)) error

Walk a list of commits with LOB references which are ancestors of 'ref' which have not been pushed Walks forwards from the oldest commit to the latest commit (including 'ref' if it includes LOBs) Walks all ancestors including second+ parents, in topological order remoteName can be a specific remote or "*" to count pushed ton *any* remote as OK

func WalkGitCommitLOBsToPushForRefSpec 

func WalkGitCommitLOBsToPushForRefSpec(remoteName string, refspec *GitRefSpec, recheck bool, callback func(commitLOB *CommitLOBRef) (quit bool, err error)) error

Walk forwards through a list of commits with LOB references based on refspec If refspec is a range, walks that specific range of commits regardless of whether it's been pushed If not, walks forwards from the oldest ancestor of refspec.Ref1 that's not pushed to the latest commit (including 'ref' if it includes LOBs) Walks all ancestors including second+ parents, in topological order remoteName can be a specific remote or "*" to count pushed ton *any* remote as OK If recheck=true then existing pushed records are ignored (all commits are walked)

func WalkGitHistory 

func WalkGitHistory(startSHA string, callback func(currentSHA, parentSHA string) (quit bool, err error)) error

Walk first parents starting from startSHA and call callback First call will be startSHA & its parent Parent will be blank string if there are no more parents & walk will stop after Optimises internally to call Git only for batches of 50

func WritePushedState 

func WritePushedState(remoteName string, shas []string) error

Overwrite entire pushed state for a remote

Types

type CheckoutCallback 

type CheckoutCallback func(t util.ProgressCallbackType, filelob *FileLOB, err error)

Callback can report skip,transfer (on complete), error

type CommitLOBRef 

type CommitLOBRef struct {
	Commit  string
	Parents []string
	// Bare LOBs
	LobSHAs []string
	// LOBs with file names
	FileLOBs []*FileLOB
}

A record of a set of LOB shas that are associated with a commit

func GetCommitLOBsToPushForRef 

func GetCommitLOBsToPushForRef(remoteName string, ref string, recheck bool) ([]*CommitLOBRef, error)

Get a list of commits which have LOB SHAs to push, given a ref, in forward ancestry order Only commits which have LOBs associated will be returned on the assumption that when child commits are marked as pushed it will also mark the parents If the refspec is a single ref, then finds the latest ancestor we think has been pushed already for this remote and returns the LOBs referred to in that range. If recheck is true, ignores the record of the last commit we think we pushed and scans entire history (slow)

func GetCommitLOBsToPushForRefSpec 

func GetCommitLOBsToPushForRefSpec(remoteName string, refspec *GitRefSpec, recheck bool) ([]*CommitLOBRef, error)

Get a list of commits which have LOB SHAs to push, given a refspec, in forward ancestry order Only commits which have LOBs associated will be returned on the assumption that when child commits are marked as pushed it will also mark the parents If the refspec is itself a range, just queries that range for binary references If the refspec is a single ref, then finds the latest ancestor we think has been pushed already for this remote and returns the LOBs referred to in that range. If recheck is true, ignores the record of the last commit we think we pushed and scans entire history (slow)

func GetGitCommitsReferencingLOBsInRange 

func GetGitCommitsReferencingLOBsInRange(from, to string, includePaths, excludePaths []string) ([]*CommitLOBRef, error)

Returns list of commits which have LOB SHAs referenced in them, in a given commit range Commits will be in ASCENDING order (parents before children) unlike WalkGitHistory Either of from, to or both can be blank to have an unbounded range of commits based on current HEAD It is required that if both are supplied, 'from' is an ancestor of 'to' Range is exclusive of 'from' and inclusive of 'to'

func (*CommitLOBRef) String 

func (self *CommitLOBRef) String() string

type FileLOB 

type FileLOB struct {
	// Filename relative to repository root
	Filename string
	// LOB SHA
	SHA string
}

A filename & LOB SHA pair

func GetGitAllFileLOBsToCheckoutAtCommitAndRecent 

func GetGitAllFileLOBsToCheckoutAtCommitAndRecent(commit string, days int, includePaths,
	excludePaths []string) (filelobs []*FileLOB, earliestChangeCommit string, reterr error)

Get all the Filenames & LOB SHAs that you would need to have available to check out a commit, and any other ancestor of it within a number of days of that commit date (not today's date) Note that if a LOB was modified to the same SHA more than once, duplicates may appear in the return They are not routinely eliminated for performance, so perform your own dupe removal if you need it as well as a list of LOBs, returns the commit SHA of the earliest change that was included in the scan. Since this is the first *change* included (which would be removing the previous SHA), the earliest LOB SHA included is from the *parent* of this commit.

func GetGitAllFilesAndLOBsToCheckoutAtCommit 

func GetGitAllFilesAndLOBsToCheckoutAtCommit(commit string, includePaths, excludePaths []string) ([]*FileLOB, error)

Get all the binary files & their LOB SHAs that you would need to check out at a given commit (not changed in that commit)

func GetGitAllFilesAndLOBsToCheckoutInRefSpec 

func GetGitAllFilesAndLOBsToCheckoutInRefSpec(refspec *GitRefSpec, includePaths, excludePaths []string) ([]*FileLOB, error)

Gets a list of LOB SHAs with their filenames for all binary files that are needed when checking out any of the commits referred to by refspec. As opposed to GetGitCommitsReferencingLOBsInRange which only picks up changes to LOBs, this function returns the complete set of LOBs needed if you checked out a commit either at a single commit, or any in a range (if the refspec is a range; only .. range operator allowed) This means it will include any LOBs that were added in commits before the range, if they are still used, while GetGitCommitsReferencingLOBsInRange wouldn't mention those. Note that git ranges are start AND end inclusive in this case. Note that duplicate SHAs are not eliminated for efficiency, you must do it if you need it

type FsckCallbackData 

type FsckCallbackData struct {
	// What stage of the process this is for, preparing, transferring or skipping something
	Type FsckCallbackType
	// What SHA is being checked
	SHA string
	// Either a general message or an item name (e.g. file name)
	Desc string
	// The percentage complete
	PercentComplete int
}

Collected callback data for a fsck operation

type FsckCallbackType 

type FsckCallbackType int
const (
	// Process is just working through data (progress update)
	FsckWorking FsckCallbackType = iota
	// A file was missing and not recovered from shared store (desc = name)
	FsckMissing FsckCallbackType = iota
	// A content file had the wrong size (desc = name)
	// This file will be deleted if --delete was specified
	FsckWrongSize FsckCallbackType = iota
	// A binary is corrupt - either the metadata is invalid, or the SHA doesn't match the content (only detected with --deep) (desc = SHA)
	// The files will be deleted if --delete was specified
	FsckCorruptData FsckCallbackType = iota
)

type GitCommitSummary 

type GitCommitSummary struct {
	SHA            string
	ShortSHA       string
	Parents        []string
	CommitDate     time.Time
	AuthorDate     time.Time
	AuthorName     string
	AuthorEmail    string
	CommitterName  string
	CommitterEmail string
	Subject        string
}

Some top level information about a commit (only first line of message)

func GetGitCommitSummary 

func GetGitCommitSummary(commit string) (*GitCommitSummary, error)

Get summary information about a commit

func GetGitLatestLOBChangeDetails 

func GetGitLatestLOBChangeDetails(filename, ref string) (summary *GitCommitSummary, lobsha string, err error)

Gets the latest change to a specific LOB file at ref, returning the SHA and the commit details

type GitRef 

type GitRef struct {
	Name      string
	Type      GitRefType
	CommitSHA string
}

A git reference (branch, tag etc)

func GetGitAllRefs 

func GetGitAllRefs() ([]*GitRef, error)

get all refs in the repo (branches, tags, stashes)

func GetGitRecentRefs 

func GetGitRecentRefs(numdays int, includeRemoteBranches bool, remoteName string) ([]*GitRef, error)

Get a list of refs (branches, tags) that have received commits in the last numdays, ordered by most recent first You can also set numdays to -1 to not have any limit but still get them in reverse order remoteName is optional but if specified and includeRemoteBranches is true, will only include remote branches on that remote

type GitRefSpec 

type GitRefSpec struct {
	// First ref
	Ref1 string
	// Optional range operator if this is a range refspec (".." or "...")
	RangeOp string
	// Optional second ref
	Ref2 string
}

Git specification of a commit or range of commits (a reference or reference range)

func ParseGitRefSpec 

func ParseGitRefSpec(s string) *GitRefSpec

Parse a single git refspec string into a GitRefSpec structure ie identify ranges if present Does not perform any validation since refs can be symbolic anyway, up to the caller to check whether the returned refspec actually works

func (*GitRefSpec) IsEmptyRange 

func (r *GitRefSpec) IsEmptyRange() bool

Returns whether a GitRefSpec is an empty range (using the same ref for start & end)

func (*GitRefSpec) IsRange 

func (r *GitRefSpec) IsRange() bool

Returns whether a GitRefSpec is a range or not

func (*GitRefSpec) String 

func (r *GitRefSpec) String() string

type GitRefType 

type GitRefType int

func ParseGitRefToTypeAndName 

func ParseGitRefToTypeAndName(fullref string) (t GitRefType, name string)

Get the type & name of a git reference

type IntegrityError 

type IntegrityError struct {
	FailedSHAs        []string
	AdditionalMessage string
}

Custom error type to indicate an integrity error, listing problem SHAs

func (*IntegrityError) Error 

func (i *IntegrityError) Error() string

type LOBDelta 

type LOBDelta struct {
	BaseSHA, TargetSHA string
	DeltaSize          int64
	// Optional already present delta filename, can be blank
	DeltaFilename string
}

Record of a LOB delta (calculated but still to be done)

type LOBInfo 

type LOBInfo struct {
	// SHA of the LOB
	SHA string
	// Total size of the LOB (all chunks)
	Size int64
	// Number of chunks that make up the whole LOB (integrity check)
	NumChunks int
}

Information about a LOB

func GetLOBInfo 

func GetLOBInfo(sha string) (*LOBInfo, error)

Retrieve information about an existing stored LOB (local)

func RetrieveLOB 

func RetrieveLOB(sha string, out io.Writer) (info *LOBInfo, err error)

Retrieve LOB from storage

func StoreLOB 

func StoreLOB(in io.Reader, leader []byte) (*LOBInfo, error)

Read from a stream and calculate SHA, while also writing content to chunked content leader is a slice of bytes that has already been read (probe for SHA)

func StoreLOBInBaseDir 

func StoreLOBInBaseDir(basedir string, in io.Reader, leader []byte) (*LOBInfo, error)

Read from a stream and calculate SHA, while also writing content to chunked content leader is a slice of bytes that has already been read (probe for SHA) Store underneath a specified LOB root

type MissingCallbackData 

type MissingCallbackData struct {
	// What stage of the process this is for
	Type MissingCallbackType
	// Path to the file (relative to working dir)
	Path string
	// Commit summary for MissingBlamed
	CommitSummary *GitCommitSummary
	// Error details for MissingError
	Error error
}

Collected callback data for a missing operation

type MissingCallbackType 

type MissingCallbackType int
const (
	// Process is just working through data (progress update)
	MissingWorking MissingCallbackType = iota
	// Placeholder present but content is available (not checked out)
	MissingAvailable MissingCallbackType = iota
	// Placeholder WAS present, content available & checked out
	MissingFixed MissingCallbackType = iota
	// Placeholder present, content corrupted (use fsck)
	MissingCorrupt MissingCallbackType = iota
	// Placeholder present, content not available (commit details included)
	MissingBlamed MissingCallbackType = iota
	// Placeholder present, but file is untracked or modified & therefore it's local user's fault
	// resolution is probably to delete or reset/checkout this file again
	MissingModified MissingCallbackType = iota
	// Some other error was encountered
	MissingError MissingCallbackType = iota
)

type NotFoundError 

type NotFoundError struct {
	Message string
	Path    string
}

Custom error type to indicate a 'not found' condition This type of error may be expected or tolerable so identify separately

func (*NotFoundError) Error 

func (i *NotFoundError) Error() string

type PruneCallback 

type PruneCallback func(t PruneCallbackType, lobsha string)

Callback when running prune, identifies what's going on When in dry run mode the same callbacks are made even if the actual act isn't performed (e.g. deletion)

type PruneCallbackType 

type PruneCallbackType int
const (
	// Prune is working (for spinner)
	PruneWorking PruneCallbackType = iota
	// Prune is retaining LOB because referenced
	PruneRetainReferenced PruneCallbackType = iota
	// Prune is retaining LOB because commit referencing it is within retention period
	PruneRetainByDate PruneCallbackType = iota
	// Prune is retaining LOB because commit is referencing it is not pushed
	PruneRetainNotPushed PruneCallbackType = iota
	// Prune is deleting LOB (because unreferenced or out of date range & pushed)
	PruneDeleted PruneCallbackType = iota
)

type PushCommitContentDetails 

type PushCommitContentDetails struct {
	CommitSHA  string      // the commit's SHA
	Deltas     []*LOBDelta // delta uploads, items in here won't be in Files
	Files      []string    // list of files we'll need to upload, relative path, if not doing deltas
	BaseDir    string      // the base dir of the above files
	FileBytes  int64       // total bytes for all files in the list
	DeltaBytes int64       // total bytes for all deltas in the list
	Incomplete bool        // File list is not complete because of missing local data, we shouldn't mark this commit as pushed
}

Some temporary storage used to pre-calculate the amount of data we'll need to upload

type WrongSizeError 

type WrongSizeError struct {
	Message  string
	Filename string
}

Custom error type to indicate a 'wrong size' condition This means a file is on disk but is the wrong size

func (*WrongSizeError) Error 

func (i *WrongSizeError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.