accounting

package
v1.48.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2019 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package accounting providers an accounting and limiting reader

Index

Constants

This section is empty.

Variables

View Source 
var ErrorMaxTransferLimitReached = fserrors.FatalError(errors.New("Max transfer limit reached as set by --max-transfer"))

ErrorMaxTransferLimitReached is returned from Read when the max transfer limit is reached.

View Source 
var (
	// Stats is global statistics counter
	Stats = NewStats()
)

Functions

func SetBwLimit 

func SetBwLimit(bandwidth fs.SizeSuffix)

SetBwLimit sets the current bandwidth limit

func StartTokenBucket 

func StartTokenBucket()

StartTokenBucket starts the token bucket if necessary

func StartTokenTicker 

func StartTokenTicker()

StartTokenTicker creates a ticker to update the bandwidth limiter every minute.

Types

type Account 

type Account struct {
	// contains filtered or unexported fields
}

Account limits and accounts for one transfer

func NewAccount 

func NewAccount(in io.ReadCloser, obj fs.Object) *Account

NewAccount makes a Account reader for an object

func NewAccountSizeName 

func NewAccountSizeName(in io.ReadCloser, size int64, name string) *Account

NewAccountSizeName makes a Account reader for an io.ReadCloser of the given size and name

func (*Account) AccountRead added in v1.48.0 

func (acc *Account) AccountRead(n int) (err error)

AccountRead account having read n bytes

func (*Account) Close 

func (acc *Account) Close() error

Close the object

func (*Account) GetAsyncReader 

func (acc *Account) GetAsyncReader() *asyncreader.AsyncReader

GetAsyncReader returns the current AsyncReader or nil if Account is unbuffered

func (*Account) GetReader 

func (acc *Account) GetReader() io.ReadCloser

GetReader returns the underlying io.ReadCloser under any Buffer

func (*Account) OldStream 

func (acc *Account) OldStream() io.Reader

OldStream returns the top io.Reader

func (*Account) Read 

func (acc *Account) Read(p []byte) (n int, err error)

Read bytes from the object - see io.Reader

func (*Account) RemoteStats 

func (acc *Account) RemoteStats() (out map[string]interface{})

RemoteStats produces stats for this file

func (*Account) SetStream 

func (acc *Account) SetStream(in io.Reader)

SetStream updates the top io.Reader

func (*Account) StopBuffering 

func (acc *Account) StopBuffering()

StopBuffering stops the async buffer doing any more buffering

func (*Account) String 

func (acc *Account) String() string

String produces stats for this file

func (*Account) UpdateReader 

func (acc *Account) UpdateReader(in io.ReadCloser)

UpdateReader updates the underlying io.ReadCloser stopping the async buffer (if any) and re-adding it

func (*Account) WithBuffer 

func (acc *Account) WithBuffer() *Account

WithBuffer - If the file is above a certain size it adds an Async reader

func (*Account) WrapStream 

func (acc *Account) WrapStream(in io.Reader) io.Reader

WrapStream wraps an io Reader so it will be accounted in the same way as account

type Accounter 

type Accounter interface {
	io.Reader
	OldStream() io.Reader
	SetStream(io.Reader)
	WrapStream(io.Reader) io.Reader
}

Accounter accounts a stream allowing the accounting to be removed and re-added

type StatsInfo 

type StatsInfo struct {
	// contains filtered or unexported fields
}

StatsInfo accounts all transfers

func NewStats 

func NewStats() *StatsInfo

NewStats creates an initialised StatsInfo

func (*StatsInfo) Bytes 

func (s *StatsInfo) Bytes(bytes int64)

Bytes updates the stats for bytes bytes

func (*StatsInfo) Checking 

func (s *StatsInfo) Checking(remote string)

Checking adds a check into the stats

func (*StatsInfo) Deletes 

func (s *StatsInfo) Deletes(deletes int64) int64

Deletes updates the stats for deletes

func (*StatsInfo) DoneChecking 

func (s *StatsInfo) DoneChecking(remote string)

DoneChecking removes a check from the stats

func (*StatsInfo) DoneTransferring 

func (s *StatsInfo) DoneTransferring(remote string, ok bool)

DoneTransferring removes a transfer from the stats

if ok is true then it increments the transfers count

func (*StatsInfo) Error 

func (s *StatsInfo) Error(err error)

Error adds a single error into the stats, assigns lastError and eventually sets fatalError or retryError

func (*StatsInfo) Errored 

func (s *StatsInfo) Errored() bool

Errored returns whether there have been any errors

func (*StatsInfo) Errors 

func (s *StatsInfo) Errors(errors int64)

Errors updates the stats for errors

func (*StatsInfo) FatalError added in v1.46.0 

func (s *StatsInfo) FatalError()

FatalError sets the fatalError flag

func (*StatsInfo) GetBytes 

func (s *StatsInfo) GetBytes() int64

GetBytes returns the number of bytes transferred so far

func (*StatsInfo) GetChecks added in v1.46.0 

func (s *StatsInfo) GetChecks() int64

GetChecks returns the number of checks

func (*StatsInfo) GetErrors 

func (s *StatsInfo) GetErrors() int64

GetErrors reads the number of errors

func (*StatsInfo) GetLastError 

func (s *StatsInfo) GetLastError() error

GetLastError returns the lastError

func (*StatsInfo) GetTransfers 

func (s *StatsInfo) GetTransfers() int64

GetTransfers reads the number of transfers

func (*StatsInfo) HadFatalError added in v1.46.0 

func (s *StatsInfo) HadFatalError() bool

HadFatalError returns whether there has been at least one FatalError

func (*StatsInfo) HadRetryError added in v1.46.0 

func (s *StatsInfo) HadRetryError() bool

HadRetryError returns whether there has been at least one non-NoRetryError

func (*StatsInfo) Log 

func (s *StatsInfo) Log()

Log outputs the StatsInfo to the log

func (*StatsInfo) RemoteStats 

func (s *StatsInfo) RemoteStats(in rc.Params) (out rc.Params, err error)

RemoteStats returns stats for rc

func (*StatsInfo) ResetCounters 

func (s *StatsInfo) ResetCounters()

ResetCounters sets the counters (bytes, checks, errors, transfers, deletes) to 0 and resets lastError, fatalError and retryError

func (*StatsInfo) ResetErrors 

func (s *StatsInfo) ResetErrors()

ResetErrors sets the errors count to 0 and resets lastError, fatalError and retryError

func (*StatsInfo) RetryAfter added in v1.47.0 

func (s *StatsInfo) RetryAfter() time.Time

RetryAfter returns the time to retry after if it is set. It will be Zero if it isn't set.

func (*StatsInfo) RetryError added in v1.46.0 

func (s *StatsInfo) RetryError()

RetryError sets the retryError flag

func (*StatsInfo) SetCheckQueue 

func (s *StatsInfo) SetCheckQueue(n int, size int64)

SetCheckQueue sets the number of queued checks

func (*StatsInfo) SetRenameQueue 

func (s *StatsInfo) SetRenameQueue(n int, size int64)

SetRenameQueue sets the number of queued transfers

func (*StatsInfo) SetTransferQueue 

func (s *StatsInfo) SetTransferQueue(n int, size int64)

SetTransferQueue sets the number of queued transfers

func (*StatsInfo) String 

func (s *StatsInfo) String() string

String convert the StatsInfo to a string for printing

func (*StatsInfo) Transferring 

func (s *StatsInfo) Transferring(remote string)

Transferring adds a transfer into the stats

type WrapFn 

type WrapFn func(io.Reader) io.Reader

WrapFn wraps an io.Reader (for accounting purposes usually)

func UnWrap 

func UnWrap(in io.Reader) (unwrapped io.Reader, wrap WrapFn)

UnWrap unwraps a reader returning unwrapped and wrap, a function to wrap it back up again. If `in` is an Accounter then this function will take the accounting unwrapped and wrap will put it back on again the new Reader passed in.

This allows functions which wrap io.Readers to move the accounting to the end of the wrapped chain of readers. This is very important if buffering is being introduced and if the Reader might be wrapped again.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.