utils: github.com/juju/utils Index | Examples | Files | Directories

package utils

import "github.com/juju/utils"

Index

Examples

Package Files

attempt.go command.go context.go file.go file_unix.go gomaxprocs.go home_unix.go http-1_7.go http.go isubuntu.go limiter.go multireader.go naturalsort.go network.go os.go password.go randomstring.go relativeurl.go setenv.go size.go systemerrmessages_unix.go timeit.go timer.go tls.go trivial.go username.go uuid.go yaml.go

Constants

const (
    // VerifySSLHostnames ensures we verify the hostname on the certificate
    // matches the host we are connecting and is signed
    VerifySSLHostnames = SSLHostnameVerification(true)
    // NoVerifySSLHostnames informs us to skip verifying the hostname
    // matches a valid certificate
    NoVerifySSLHostnames = SSLHostnameVerification(false)
)
const (
    OSWindows   = "windows"
    OSDarwin    = "darwin"
    OSDragonfly = "dragonfly"
    OSFreebsd   = "freebsd"
    OSLinux     = "linux"
    OSNacl      = "nacl"
    OSNetbsd    = "netbsd"
    OSOpenbsd   = "openbsd"
    OSSolaris   = "solaris"
)

These are the names of the operating systems recognized by Go.

const (
    NoSuchUserErrRegexp = `user: unknown user [a-z0-9_-]*`
    NoSuchFileErrRegexp = `no such file or directory`
    MkdirFailErrRegexp  = `.* not a directory`
)

The following are strings/regex-es which match common Unix error messages that may be returned in case of failed calls to the system. Any extra leading/trailing regex-es are left to be added by the developer.

Variables

var (
    LowerAlpha = []rune("abcdefghijklmnopqrstuvwxyz")
    UpperAlpha = []rune("ABCDEFGHIJKLMNOPQRSTUVWXYZ")
    Digits     = []rune("0123456789")
)

Can be used as a sane default argument for RandomString

var CompatSalt = string([]byte{0x75, 0x82, 0x81, 0xca})

CompatSalt is because Juju 1.16 and older used a hard-coded salt to compute the password hash for all users and agents

var FastInsecureHash = false

FastInsecureHash specifies whether a fast, insecure version of the hash algorithm will be used. Changing this will cause PasswordHash to produce incompatible passwords. It should only be changed for testing purposes - to make tests run faster.

var MinAgentPasswordLength = base64.StdEncoding.EncodedLen(randomPasswordBytes)

MinAgentPasswordLength describes how long agent passwords should be. We require this length because we assume enough entropy in the Agent password that it is safe to not do extra rounds of iterated hashing.

var OSUnix = []string{
    OSDarwin,
    OSDragonfly,
    OSFreebsd,
    OSLinux,
    OSNacl,
    OSNetbsd,
    OSOpenbsd,
    OSSolaris,
}

OSUnix is the list of unix-like operating systems recognized by Go. See http://golang.org/src/path/filepath/path_unix.go.

var OutgoingAccessAllowed = true

OutgoingAccessAllowed determines whether connections other than localhost can be dialled.

var (
    UUIDSnippet = block1 + "-" + block2 + "-" + block3 + "-" + block4 + "-" + block5
)

regex for validating that the UUID matches RFC 4122. This package generates version 4 UUIDs but accepts any UUID version. http://www.ietf.org/rfc/rfc4122.txt

func AgentPasswordHash Uses

func AgentPasswordHash(password string) string

AgentPasswordHash returns base64-encoded one-way hash of password. This is not suitable for User passwords because those will have limited entropy (see UserPasswordHash). However, since we generate long random passwords for agents, we can trust that there is sufficient entropy to prevent brute force search. And using a faster hash allows us to restart the state machines and have 1000s of agents log in in a reasonable amount of time.

func AtomicWriteFile Uses

func AtomicWriteFile(filename string, contents []byte, perms os.FileMode) (err error)

AtomicWriteFile atomically writes the filename with the given contents and permissions, replacing any existing file at the same path.

func AtomicWriteFileAndChange Uses

func AtomicWriteFileAndChange(filename string, contents []byte, change func(string) error) (err error)

AtomicWriteFileAndChange atomically writes the filename with the given contents and calls the given function after the contents were written, but before the file is renamed.

func BasicAuthHeader Uses

func BasicAuthHeader(username, password string) http.Header

BasicAuthHeader creates a header that contains just the "Authorization" entry. The implementation was originally taked from net/http but this is needed externally from the http request object in order to use this with our websockets. See 2 (end of page 4) http://www.ietf.org/rfc/rfc2617.txt "To receive authorization, the client sends the userid and password, separated by a single colon (":") character, within a base64 encoded string in the credentials."

func ChownPath Uses

func ChownPath(path, username string) error

ChownPath sets the uid and gid of path to match that of the user specified.

func CommandString Uses

func CommandString(args ...string) string

CommandString flattens a sequence of command arguments into a string suitable for executing in a shell, escaping slashes, variables and quotes as necessary; each argument is double-quoted if and only if necessary.

func ConformYAML Uses

func ConformYAML(input interface{}) (interface{}, error)

ConformYAML ensures all keys of any nested maps are strings. This is necessary because YAML unmarshals map[interface{}]interface{} in nested maps, which cannot be serialized by json or bson. Also, handle []interface{}. cf. gopkg.in/juju/charm.v4/actions.go cleanse

func ContextWithDeadline Uses

func ContextWithDeadline(parent context.Context, clk clock.Clock, deadline time.Time) (context.Context, context.CancelFunc)

ContextWithDeadline is like context.WithDeadline except that it works with a clock.Clock rather than wall-clock time.

func ContextWithTimeout Uses

func ContextWithTimeout(parent context.Context, clk clock.Clock, timeout time.Duration) (context.Context, context.CancelFunc)

ContextWithTimeout is like context.WithTimeout except that it works with a clock.Clock rather than wall-clock time.

func CopyFile Uses

func CopyFile(dest, source string) error

CopyFile writes the contents of the given source file to dest.

func EnsureBaseDir Uses

func EnsureBaseDir(baseDir, path string) string

EnsureBaseDir ensures that path is always prefixed by baseDir, allowing for the fact that path might have a Window drive letter in it.

func EnvUsername Uses

func EnvUsername() (string, error)

EnvUsername returns the username from the OS environment.

func ExpandPath Uses

func ExpandPath(path string) (string, error)

ExpandPath normalises (via Normalize) a path returning an absolute path.

func GetAddressForInterface Uses

func GetAddressForInterface(interfaceName string) (string, error)

GetAddressForInterface looks for the network interface and returns the IPv4 address from the possible addresses.

func GetHTTPClient Uses

func GetHTTPClient(verify SSLHostnameVerification) *http.Client

GetHTTPClient returns either a standard http client or non validating client depending on the value of verify.

func GetIPv4Address Uses

func GetIPv4Address(addresses []net.Addr) (string, error)

GetIPv4Address iterates through the addresses expecting the format from func (ifi *net.Interface) Addrs() ([]net.Addr, error)

func GetIPv6Address Uses

func GetIPv6Address(addresses []net.Addr) (string, error)

GetIPv6Address iterates through the addresses expecting the format from func (ifi *net.Interface) Addrs() ([]net.Addr, error) and returns the first non-link local address.

func GetNonValidatingHTTPClient Uses

func GetNonValidatingHTTPClient() *http.Client

GetNonValidatingHTTPClient returns a new http.Client that does not verify the server's certificate chain and hostname.

func GetV4OrV6AddressForInterface Uses

func GetV4OrV6AddressForInterface(interfaceName string) (string, error)

GetV4OrV6AddressForInterface looks for the network interface and returns preferably the IPv4 address, and if it doesn't exists then IPv6 address.

func GetValidatingHTTPClient Uses

func GetValidatingHTTPClient() *http.Client

GetValidatingHTTPClient returns a new http.Client that verifies the server's certificate chain and hostname.

func Gunzip Uses

func Gunzip(data []byte) ([]byte, error)

Gunzip uncompresses the given data.

func Gzip Uses

func Gzip(data []byte) []byte

Gzip compresses the given data.

func Home Uses

func Home() string

Home returns the os-specific home path as specified in the environment.

func IsFileOwner Uses

func IsFileOwner(path, username string) (bool, error)

IsFileOwner checks to see if the ownership of the file corresponds to the same username

func IsUbuntu Uses

func IsUbuntu() bool

IsUbuntu executes lxb_release to see if the host OS is Ubuntu.

func IsValidUUIDString Uses

func IsValidUUIDString(s string) bool

IsValidUUIDString returns true, if the given string matches a valid UUID (version 4, variant 2).

func JoinServerPath Uses

func JoinServerPath(elem ...string) string

JoinServerPath joins any number of path elements into a single path, adding a path separator (based on the current juju server OS) if necessary. The result is Cleaned; in particular, all empty strings are ignored.

func LocalUsername Uses

func LocalUsername() (string, error)

LocalUsername determines the current username on the local host.

func MakeFileURL Uses

func MakeFileURL(in string) string

MakeFileURL returns a file URL if a directory is passed in else it does nothing

func MoveFile Uses

func MoveFile(source, destination string) (bool, error)

MoveFile atomically moves the source file to the destination, returning whether the file was moved successfully. If the destination already exists, it returns an error rather than overwrite it.

On unix systems, an error may occur with a successful move, if the source file location cannot be unlinked.

func NewHttpTLSTransport Uses

func NewHttpTLSTransport(tlsConfig *tls.Config) *http.Transport

NewHttpTLSTransport returns a new http.Transport constructed with the TLS config and the necessary parameters for Juju.

func NewMultiReaderSeeker Uses

func NewMultiReaderSeeker(readers ...io.ReadSeeker) io.ReadSeeker

NewMultiReaderSeeker returns an io.ReadSeeker that combines all the given readers into a single one. It assumes that all the seekers are initially positioned at the start.

func NormalizePath Uses

func NormalizePath(dir string) (string, error)

NormalizePath expands a path containing ~ to its absolute form, and removes any .. or . path elements.

func OSIsUnix Uses

func OSIsUnix(os string) bool

OSIsUnix determines whether or not the given OS name is one of the unix-like operating systems recognized by Go.

func OSUsername Uses

func OSUsername() (string, error)

OSUsername returns the username of the current OS user (based on UID).

func ParseBasicAuthHeader Uses

func ParseBasicAuthHeader(h http.Header) (userid, password string, err error)

ParseBasicAuth attempts to find an Authorization header in the supplied http.Header and if found parses it as a Basic header. See 2 (end of page 4) http://www.ietf.org/rfc/rfc2617.txt "To receive authorization, the client sends the userid and password, separated by a single colon (":") character, within a base64 encoded string in the credentials."

func ParseSize Uses

func ParseSize(str string) (MB uint64, err error)

ParseSize parses the string as a size, in mebibytes.

The string must be a is a non-negative number with an optional multiplier suffix (M, G, T, P, E, Z, or Y). If the suffix is not specified, "M" is implied.

func RandomBytes Uses

func RandomBytes(n int) ([]byte, error)

RandomBytes returns n random bytes.

func RandomPassword Uses

func RandomPassword() (string, error)

RandomPassword generates a random base64-encoded password.

func RandomSalt Uses

func RandomSalt() (string, error)

RandomSalt generates a random base64 data suitable for using as a password salt The pbkdf2 guideline is to use 8 bytes of salt, so we do 12 raw bytes into 16 base64 bytes. (The alternative is 6 raw into 8 base64).

func RandomString Uses

func RandomString(n int, validRunes []rune) string

RandomString will return a string of length n that will only contain runes inside validRunes

func ReadFileSHA256 Uses

func ReadFileSHA256(filename string) (string, int64, error)

ReadFileSHA256 is like ReadSHA256 but reads the contents of the given file.

func ReadSHA256 Uses

func ReadSHA256(source io.Reader) (string, int64, error)

ReadSHA256 returns the SHA256 hash of the contents read from source (hex encoded) and the size of the source in bytes.

func ReadYaml Uses

func ReadYaml(path string, obj interface{}) error

ReadYaml unmarshals the yaml contained in the file at path into obj. See goyaml.Unmarshal. If path is not found, the error returned will be compatible with os.IsNotExist.

func RelativeURLPath Uses

func RelativeURLPath(basePath, targPath string) (string, error)

RelativeURLPath returns a relative URL path that is lexically equivalent to targpath when interpreted by url.URL.ResolveReference. On success, the returned path will always be non-empty and relative to basePath, even if basePath and targPath share no elements.

It is assumed that both basePath and targPath are normalized (have no . or .. elements).

An error is returned if basePath or targPath are not absolute paths.

func ReplaceFile Uses

func ReplaceFile(source, destination string) error

ReplaceFile atomically replaces the destination file or directory with the source. The errors that are returned are identical to those returned by os.Rename.

func ResolveSudo Uses

func ResolveSudo(username string) string

ResolveSudo returns the original username if sudo was used. The original username is extracted from the OS environment.

func ResolveUsername Uses

func ResolveUsername(resolveSudo func(string) string, usernameFuncs ...func() (string, error)) (string, error)

ResolveUsername returns the username determined by the provided functions. The functions are tried in the same order in which they were passed in. An error returned from any of them is immediately returned. If an empty string is returned then that signals that the function did not find the username and the next function is tried. Once a username is found, the provided resolveSudo func (if any) is called with that username and the result is returned. If no username is found then errors.NotFound is returned.

func RunCommand Uses

func RunCommand(command string, args ...string) (output string, err error)

RunCommand executes the command and return the combined output.

func SecureTLSConfig Uses

func SecureTLSConfig() *tls.Config

SecureTLSConfig returns a tls.Config that conforms to Juju's security standards, so as to avoid known security vulnerabilities in certain configurations.

Currently it excludes RC4 implementations from the available ciphersuites, requires ciphersuites that provide forward secrecy, and sets the minimum TLS version to 1.2.

func SetHome Uses

func SetHome(s string) error

SetHome sets the os-specific home path in the environment.

func Setenv Uses

func Setenv(env []string, entry string) []string

Setenv sets an environment variable entry in the given env slice (as returned by os.Environ or passed in exec.Cmd.Environ) to the given value. The entry should be in the form "x=y" where x is the name of the environment variable and y is its value; if not, env will be returned unchanged.

If a value isn't already present in the slice, the entry is appended.

The new environ slice is returned.

func ShQuote Uses

func ShQuote(s string) string

ShQuote quotes s so that when read by bash, no metacharacters within s will be interpreted as such.

func SortStringsNaturally Uses

func SortStringsNaturally(s []string) []string

SortStringsNaturally sorts strings according to their natural sort order.

func Timeit Uses

func Timeit(action string) func()

Start a timer, used for tracking time spent. Generally used with either defer, as in:

defer utils.Timeit("my func")()

Which will track how much time is spent in your function. Or if you want to track the time spent in a function you are calling then you would use:

toc := utils.Timeit("anotherFunc()")
anotherFunc()
toc()

This tracks nested calls by indenting the output, and will print out the full stack of timing when we reach the top of the stack.

func UniqueDirectory Uses

func UniqueDirectory(path, name string) (string, error)

UniqueDirectory returns "path/name" if that directory doesn't exist. If it does, the method starts appending .1, .2, etc until a unique name is found.

func UseMultipleCPUs Uses

func UseMultipleCPUs()

UseMultipleCPUs sets GOMAXPROCS to the number of CPU cores unless it has already been overridden by the GOMAXPROCS environment variable.

func UserHomeDir Uses

func UserHomeDir(userName string) (hDir string, err error)

UserHomeDir returns the home directory for the specified user, or the home directory for the current user if the specified user is empty.

func UserPasswordHash Uses

func UserPasswordHash(password string, salt string) string

UserPasswordHash returns base64-encoded one-way hash password that is computationally hard to crack by iterating through possible passwords.

func WinCmdQuote Uses

func WinCmdQuote(s string) string

WinCmdQuote quotes s so that when read by cmd.exe, no metacharacters within s will be interpreted as such.

func WinPSQuote Uses

func WinPSQuote(s string) string

WinPSQuote quotes s so that when read by powershell, no metacharacters within s will be interpreted as such.

func WriteYaml Uses

func WriteYaml(path string, obj interface{}) error

WriteYaml marshals obj as yaml to a temporary file in the same directory as path, than atomically replaces path with the temporary file.

type Attempt Uses

type Attempt struct {
    // contains filtered or unexported fields
}

func (*Attempt) HasNext Uses

func (a *Attempt) HasNext() bool

HasNext returns whether another attempt will be made if the current one fails. If it returns true, the following call to Next is guaranteed to return true.

Code:

// This example shows how Attempt.HasNext can be used to help
// structure an attempt loop. If the godoc example code allowed
// us to make the example return an error, we would uncomment
// the commented return statements.
attempts := utils.AttemptStrategy{
    Total: 1 * time.Second,
    Delay: 250 * time.Millisecond,
}
for attempt := attempts.Start(); attempt.Next(); {
    x, err := doSomething()
    if shouldRetry(err) && attempt.HasNext() {
        continue
    }
    if err != nil {
        // return err
        return
    }
    doSomethingWith(x)
}
// return ErrTimedOut
return

func (*Attempt) Next Uses

func (a *Attempt) Next() bool

Next waits until it is time to perform the next attempt or returns false if it is time to stop trying. It always returns true the first time it is called - we are guaranteed to make at least one attempt.

type AttemptStrategy Uses

type AttemptStrategy struct {
    Total time.Duration // total duration of attempt.
    Delay time.Duration // interval between each try in the burst.
    Min   int           // minimum number of retries; overrides Total
}

AttemptStrategy represents a strategy for waiting for an action to complete successfully.

func (AttemptStrategy) Start Uses

func (s AttemptStrategy) Start() *Attempt

Start begins a new sequence of attempts for the given strategy.

type BackoffTimer Uses

type BackoffTimer struct {
    // contains filtered or unexported fields
}

BackoffTimer implements Countdown. A backoff timer starts at min and gets multiplied by factor until it reaches max. Jitter determines whether a small randomization is added to the duration.

TODO(katco): 2016-08-09: This type is deprecated: lp:1611427

func NewBackoffTimer Uses

func NewBackoffTimer(config BackoffTimerConfig) *BackoffTimer

NewBackoffTimer creates and initializes a new BackoffTimer A backoff timer starts at min and gets multiplied by factor until it reaches max. Jitter determines whether a small randomization is added to the duration.

TODO(katco): 2016-08-09: This type is deprecated: lp:1611427

func (*BackoffTimer) Reset Uses

func (t *BackoffTimer) Reset()

Reset implements the Timer interface.

func (*BackoffTimer) Start Uses

func (t *BackoffTimer) Start()

Start implements the Timer interface. Any existing timer execution is stopped before a new one is created.

type BackoffTimerConfig Uses

type BackoffTimerConfig struct {
    // The minimum duration after which Func is called.
    Min time.Duration

    // The maximum duration after which Func is called.
    Max time.Duration

    // Determines whether a small randomization is applied to
    // the duration.
    Jitter bool

    // The factor by which you want the duration to increase
    // every time.
    Factor int64

    // Func is the function that will be called when the countdown reaches 0.
    Func func()

    // Clock provides the AfterFunc function used to call func.
    // It is exposed here so it's easier to mock it in tests.
    Clock clock.Clock
}

BackoffTimerConfig is a helper struct for backoff timer that encapsulates config information.

TODO(katco): 2016-08-09: This type is deprecated: lp:1611427

type Countdown Uses

type Countdown interface {
    // Reset stops the timer and resets its duration to the minimum one.
    // Start must be called to start the timer again.
    Reset()

    // Start starts the internal timer.
    // At the end of the timer, if Reset hasn't been called in the mean time
    // Func will be called and the duration is increased for the next call.
    Start()
}

Countdown implements a timer that will call a provided function. after a internally stored duration. The steps as well as min and max durations are declared upon initialization and depend on the particular implementation.

TODO(katco): 2016-08-09: This type is deprecated: lp:1611427

type Limiter Uses

type Limiter interface {
    // Acquire another unit of the resource.
    // Acquire returns false to indicate there is no more availability,
    // until another entity calls Release.
    Acquire() bool
    // AcquireWait requests a unit of resource, but blocks until one is
    // available.
    AcquireWait()
    // Release returns a unit of the resource. Calling Release when there
    // are no units Acquired is an error.
    Release() error
}

Limiter represents a limited resource (eg a semaphore).

func NewLimiter Uses

func NewLimiter(maxAllowed int) Limiter

NewLimiter creates a limiter.

func NewLimiterWithPause Uses

func NewLimiterWithPause(maxAllowed int, minPause, maxPause time.Duration, clk clock.Clock) Limiter

NewLimiterWithPause creates a limiter. If minpause and maxPause is > 0, there will be a random delay in that duration range before attempting an Acquire.

type SSLHostnameVerification Uses

type SSLHostnameVerification bool

SSLHostnameVerification is used as a switch for when a given provider might use self-signed credentials and we should not try to verify the hostname on the TLS/SSL certificates

type SizeReaderAt Uses

type SizeReaderAt interface {
    // Size returns the size of the data readable
    // from the reader.
    Size() int64
    io.ReaderAt
}

SizeReaderAt combines io.ReaderAt with a Size method.

func NewMultiReaderAt Uses

func NewMultiReaderAt(parts ...SizeReaderAt) SizeReaderAt

NewMultiReaderAt is like io.MultiReader but produces a ReaderAt (and Size), instead of just a reader.

Note: this implementation was taken from a talk given by Brad Fitzpatrick as OSCON 2013.

http://talks.golang.org/2013/oscon-dl.slide#49 https://github.com/golang/talks/blob/master/2013/oscon-dl/server-compose.go

type SizeTracker Uses

type SizeTracker struct {
    // contains filtered or unexported fields
}

SizeTracker tracks the number of bytes passing through its Write method (which is otherwise a no-op).

Use SizeTracker with io.MultiWriter() to track number of bytes written. Use with io.TeeReader() to track number of bytes read.

func (SizeTracker) Size Uses

func (st SizeTracker) Size() int64

Size returns the number of bytes written so far.

func (*SizeTracker) Write Uses

func (st *SizeTracker) Write(data []byte) (n int, err error)

Write implements io.Writer.

type UUID Uses

type UUID [16]byte

UUID represent a universal identifier with 16 octets.

func MustNewUUID Uses

func MustNewUUID() UUID

MustNewUUID returns a new uuid, if an error occurs it panics.

func NewUUID Uses

func NewUUID() (UUID, error)

NewUUID generates a new version 4 UUID relying only on random numbers.

func UUIDFromString Uses

func UUIDFromString(s string) (UUID, error)

func (UUID) Copy Uses

func (uuid UUID) Copy() UUID

Copy returns a copy of the UUID.

func (UUID) Raw Uses

func (uuid UUID) Raw() [16]byte

Raw returns a copy of the UUID bytes.

func (UUID) String Uses

func (uuid UUID) String() string

String returns a hexadecimal string representation with standardized separators.

Directories

PathSynopsis
arch
bzrPackage bzr offers an interface to manage branches of the Bazaar VCS.
cachePackage cache provides a simple caching mechanism that limits the age of cache entries and tries to avoid large repopulation events by staggering refresh times.
cert
debugstatusPackage debugstatus provides facilities for inspecting information about a running HTTP service.
dequeThe deque package implements an efficient double-ended queue data structure called Deque.
du
exec
featureflagThe featureflag package gives other parts of Juju the ability to easily check to see if a feature flag has been defined.
filepath
filestorageutils/filestorage provides types for abstracting and implementing a system that stores files, including their metadata.
fs
hashThe hash package provides utilities that support use of the stdlib hash.Hash.
jsonhttp
keyvaluesThe keyvalues package implements a set of functions for parsing key=value data, usually passed in as command-line parameters to juju subcommands, e.g.
mgokvPackage mgokv defines cached MongoDB-backed global persistent storage for key-value pairs.
osPackage os provides access to operating system related configuration.
parallelThe parallel package provides utilities for running tasks concurrently.
proxy
readpass
registry
seriesseries provides helpers for determining the series of a host, and translating from os to series.
set
shell
sshPackage ssh contains utilities for dealing with SSH connections, key management, and so on.
ssh/testing
symlink
tailer
tarThis package provides convenience helpers on top of archive/tar to be able to tar/untar files with a functionality closer to gnu tar command.
uptime
voyeurPackage voyeur implements a concurrency-safe value that can be watched for changes.
winrm
zip

Package utils imports 36 packages (graph) and is imported by 3693 packages. Updated 2018-08-21. Refresh now. Tools for package owners.