go-ipfs-cmds: github.com/ipfs/go-ipfs-cmds Index | Files | Directories

package cmds

import "github.com/ipfs/go-ipfs-cmds"

Package cmds helps building both standalone and client-server applications.


The basic building blocks are requests, commands, emitters and responses. A command consists of a description of the parameters and a function. The function is passed the request as well as an emitter as arguments. It does operations on the inputs and sends the results to the user by emitting them.

There are a number of emitters in this package and subpackages, but the user is free to create their own.


A command is a struct containing the commands help text, a description of the arguments and options, the command's processing function and a type to let the caller know what type will be emitted. Optionally one of the functions PostRun and Encoder may be defined that consumes the function's emitted values and generates a visual representation for e.g. the terminal. Encoders work on a value-by-value basis, while PostRun operates on the value stream.


An emitter has the Emit method, that takes the command's function's output as an argument and passes it to the user.

type ResponseEmitter interface {
	Close() error
	CloseWithError(error) error
	SetLength(length uint64)
	Emit(value interface{}) error

The command's function does not know what kind of emitter it works with, so the same function may run locally or on a server, using an rpc interface. Emitters can also send errors using the SetError method.

The user-facing emitter usually is the cli emitter. Values emitter here will be printed to the terminal using either the Encoders or the PostRun function.


A response is a value that the user can read emitted values from.

type Response interface {
	Request() Request
	Error() *Error
	Length() uint64
	Next() (interface{}, error)

Responses have a method Next() that returns the next emitted value and an error value. If the last element has been received, the returned error value is io.EOF. If the application code has sent an error using SetError, the error ErrRcvdError is returned on next, indicating that the caller should call Error(). Depending on the reponse type, other errors may also occur.


Pipes are pairs (emitter, response), such that a value emitted on the emitter can be received in the response value. Most builtin emitters are "pipe" emitters. The most prominent examples are the channel pipe and the http pipe.

The channel pipe is backed by a channel. The only error value returned by the response is io.EOF, which happens when the channel is closed.

The http pipe is backed by an http connection. The response can also return other errors, e.g. if there are errors on the network.


To get a better idea of what's going on, take a look at the examples at https://github.com/ipfs/go-ipfs-cmds/tree/master/examples.


Package Files

argument.go arguments.go chan.go channelmarshaler.go command.go doc.go encoding.go error.go executor.go flushfwd.go helptext.go option.go opts.go reqlog.go request.go response.go responseemitter.go writer.go


const (
    // General purpose
    Undefined = ""

    // EncodingTypes
    JSON        = "json"
    XML         = "xml"
    Protobuf    = "protobuf"
    Text        = "text"
    TextNewline = "textnl"

    // PostRunTypes
    CLI = "cli"

Supported EncodingType constants.

const (
    Invalid = reflect.Invalid
    Bool    = reflect.Bool
    Int     = reflect.Int
    Uint    = reflect.Uint
    Int64   = reflect.Int64
    Uint64  = reflect.Uint64
    Float   = reflect.Float64
    String  = reflect.String
    Strings = reflect.Array

Types of Command options

const (
    EncShort     = "enc"
    EncLong      = "encoding"
    RecShort     = "r"
    RecLong      = "recursive"
    ChanOpt      = "stream-channels"
    TimeoutOpt   = "timeout"
    OptShortHelp = "h"
    OptLongHelp  = "help"
    DerefLong    = "dereference-args"
    StdinName    = "stdin-name"
    Hidden       = "hidden"
    HiddenShort  = "H"
    Ignore       = "ignore"
    IgnoreRules  = "ignore-rules-path"

Flag names

const DefaultOutputEncoding = JSON

DefaultOutputEncoding defines the default API output encoding.


var (
    // ErrNotCallable signals a command that cannot be called.
    ErrNotCallable = ClientError("this command cannot be called directly; try one of its subcommands.")

    // ErrNoFormatter signals that the command can not be formatted.
    ErrNoFormatter = ClientError("this command cannot be formatted to plain text")

    // ErrIncorrectType signales that the commands returned a value with unexpected type.
    ErrIncorrectType = errors.New("the command returned a value with a different type than expected")
var (
    ErrClosedEmitter        = errors.New("cmds: emit on closed emitter")
    ErrClosingClosedEmitter = errors.New("cmds: closing closed emitter")
var Decoders = map[EncodingType]func(w io.Reader) Decoder{
    XML: func(r io.Reader) Decoder {
        return xml.NewDecoder(r)
    JSON: func(r io.Reader) Decoder {
        return json.NewDecoder(r)
var Encoders = EncoderMap{
    XML: func(req *Request) func(io.Writer) Encoder {
        return func(w io.Writer) Encoder { return xml.NewEncoder(w) }
    JSON: func(req *Request) func(io.Writer) Encoder {
        return func(w io.Writer) Encoder { return json.NewEncoder(w) }
    Text: func(req *Request) func(io.Writer) Encoder {
        return func(w io.Writer) Encoder { return TextEncoder{w: w} }
    TextNewline: func(req *Request) func(io.Writer) Encoder {
        return func(w io.Writer) Encoder { return TextEncoder{w: w, suffix: "\n"} }
var OptionDerefArgs = BoolOption(DerefLong, "Symlinks supplied in arguments are dereferenced")
var OptionEncodingType = StringOption(EncLong, EncShort, "The encoding type the output should be encoded with (json, xml, or text)").WithDefault("text")

options that are used by this package

var OptionHidden = BoolOption(Hidden, HiddenShort, "Include files that are hidden. Only takes effect on recursive add.")
var OptionIgnore = StringsOption(Ignore, "A rule (.gitignore-stype) defining which file(s) should be ignored (variadic, experimental)")
var OptionIgnoreRules = StringOption(IgnoreRules, "A path to a file with .gitignore-style ignore rules (experimental)")
var OptionRecursivePath = BoolOption(RecLong, RecShort, "Add directory paths recursively")
var OptionStdinName = StringOption(StdinName, "Assign a name if the file source is stdin.")
var OptionStreamChannels = BoolOption(ChanOpt, "Stream channel output")
var OptionTimeout = StringOption(TimeoutOpt, "Set a global timeout on the command")

func ClientError Uses

func ClientError(msg string) error

func Copy Uses

func Copy(re ResponseEmitter, res Response) error

Copy sends all values received on res to re. If res is closed, it closes re.

func EmitChan Uses

func EmitChan(re ResponseEmitter, ch <-chan interface{}) error

func EmitOnce Uses

func EmitOnce(re ResponseEmitter, v interface{}) error

EmitOnce is a helper that emits a value wrapped in Single, to signal that this will be the only value sent.

func GetEncoder Uses

func GetEncoder(req *Request, w io.Writer, def EncodingType) (encType EncodingType, enc Encoder, err error)

GetEncoder takes a request and returns returns the encoding type and the encoder.

func MakeEncoder Uses

func MakeEncoder(f func(*Request, io.Writer, interface{}) error) func(*Request) func(io.Writer) Encoder

func MakeTypedEncoder Uses

func MakeTypedEncoder(f interface{}) func(*Request) func(io.Writer) Encoder

func NewChanResponsePair Uses

func NewChanResponsePair(req *Request) (ResponseEmitter, Response)

type Argument Uses

type Argument struct {
    Name          string
    Type          ArgumentType
    Required      bool // error if no value is specified
    Variadic      bool // unlimited values can be specfied
    SupportsStdin bool // can accept stdin as a value
    Recursive     bool // supports recursive file adding (with '-r' flag)
    Description   string

func FileArg Uses

func FileArg(name string, required, variadic bool, description string) Argument

func StringArg Uses

func StringArg(name string, required, variadic bool, description string) Argument

func (Argument) EnableRecursive Uses

func (a Argument) EnableRecursive() Argument

func (Argument) EnableStdin Uses

func (a Argument) EnableStdin() Argument

type ArgumentType Uses

type ArgumentType int
const (
    ArgString ArgumentType = iota

type Command Uses

type Command struct {
    // Options defines the flags accepted by the command. Flags on specified
    // on parent commands are inherited by sub commands.
    Options []Option

    // Arguments defines the positional arguments for the command. These
    // arguments can be strings and/or files.
    // The rules for valid arguments are as follows:
    // 1. No required arguments may follow optional arguments.
    // 2. There can be at most one STDIN argument.
    // 3. There can be at most one variadic argument, and it must be last.
    Arguments []Argument

    // PreRun is run before Run. When executing a command on a remote
    // daemon, PreRun is always run in the local process.
    PreRun func(req *Request, env Environment) error

    // Run is the function that processes the request to generate a response.
    // Note that when executing the command over the HTTP API you can only read
    // after writing when using multipart requests. The request body will not be
    // available for reading after the HTTP connection has been written to.
    Run Function

    // PostRun is run after Run, and can transform results returned by run.
    // When executing a command on a remote daemon, PostRun is always run in
    // the local process.
    PostRun PostRunMap

    // Encoders encode results from Run (and/or PostRun) in the desired
    // encoding.
    Encoders EncoderMap

    // Helptext is the command's help text.
    Helptext HelpText

    // External denotes that a command is actually an external binary.
    // fewer checks and validations will be performed on such commands.
    External bool

    // Type describes the type of the output of the Command's Run Function.
    // In precise terms, the value of Type is an instance of the return type of
    // the Run Function.
    // ie. If command Run returns &Block{}, then Command.Type == &Block{}
    Type interface{}

    // Subcommands allow attaching sub commands to a command.
    // Note: A command can specify both a Run function and Subcommands. If
    // invoked with no arguments, or an argument that matches no
    // sub commands, the Run function of the current command will be invoked.
    // Take care when specifying both a Run function and Subcommands. A
    // simple typo in a sub command will invoke the parent command and may
    // end up returning a cryptic error to the user.
    Subcommands map[string]*Command

    // NoRemote denotes that a command cannot be executed in a remote environment
    NoRemote bool

    // NoLocal denotes that a command cannot be executed in a local environment
    NoLocal bool

    // Extra contains a set of other command-specific parameters
    Extra *Extra

Command is a runnable command, with input arguments and options (flags). It can also have Subcommands, to group units of work into sets.

func (*Command) Call Uses

func (c *Command) Call(req *Request, re ResponseEmitter, env Environment)

Call invokes the command for the given Request

func (*Command) CheckArguments Uses

func (c *Command) CheckArguments(req *Request) error

CheckArguments checks that we have all the required string arguments, loading any from stdin if necessary.

func (*Command) DebugValidate Uses

func (c *Command) DebugValidate() map[string][]error

DebugValidate checks if the command tree is well-formed.

This operation is slow and should be called from tests only.

func (*Command) Get Uses

func (c *Command) Get(path []string) (*Command, error)

Get resolves and returns the Command addressed by path

func (*Command) GetOptions Uses

func (c *Command) GetOptions(path []string) (map[string]Option, error)

GetOptions returns the options in the given path of commands

func (*Command) ProcessHelp Uses

func (c *Command) ProcessHelp()

func (*Command) Resolve Uses

func (c *Command) Resolve(pth []string) ([]*Command, error)

Resolve returns the subcommands at the given path The returned set of subcommands starts with this command and therefore is always at least size 1

func (*Command) Walk Uses

func (c *Command) Walk(visitor CommandVisitor)

Walks tree of all subcommands (including this one)

type CommandVisitor Uses

type CommandVisitor func(*Command)

type Decoder Uses

type Decoder interface {
    Decode(value interface{}) error

Decoder decodes values into value (which should be a pointer).

type Encoder Uses

type Encoder interface {
    Encode(value interface{}) error

Encoder encodes values onto e.g. an io.Writer. Examples are json.Encoder and xml.Encoder.

type EncoderFunc Uses

type EncoderFunc func(req *Request) func(w io.Writer) Encoder

type EncoderMap Uses

type EncoderMap map[EncodingType]EncoderFunc

type EncodingType Uses

type EncodingType string

EncodingType defines a supported encoding

func GetEncoding Uses

func GetEncoding(req *Request, def EncodingType) EncodingType

GetEncoding returns the EncodingType set in a request, falling back to JSON

type Environment Uses

type Environment interface{}

Environment is the environment passed to commands.

type Error Uses

type Error struct {
    Message string
    Code    ErrorType

Error is a struct for marshalling errors

func Errorf Uses

func Errorf(code ErrorType, format string, args ...interface{}) Error

Errorf returns an Error with the given code and format specification

func (Error) Error Uses

func (e Error) Error() string

func (Error) MarshalJSON Uses

func (e Error) MarshalJSON() ([]byte, error)

func (*Error) UnmarshalJSON Uses

func (e *Error) UnmarshalJSON(data []byte) error

func (Error) Unwrap Uses

func (e Error) Unwrap() error

Unwrap returns the base error (an ErrorType). Works with go 1.13 error helpers.

type ErrorType Uses

type ErrorType uint

ErrorType signfies a category of errors

const (
    // ErrNormal is a normal error. The command failed for some reason that's not a bug.
    ErrNormal ErrorType = iota
    // ErrClient means the client made an invalid request.
    // ErrImplementation means there's a bug in the implementation.
    // ErrRateLimited is returned when the operation has been rate-limited.
    // ErrForbidden is returned when the client doesn't have permission to
    // perform the requested operation.

ErrorTypes convey what category of error ocurred

func (ErrorType) Error Uses

func (e ErrorType) Error() string

func (ErrorType) String Uses

func (e ErrorType) String() string

type Executor Uses

type Executor interface {
    Execute(req *Request, re ResponseEmitter, env Environment) error

func NewExecutor Uses

func NewExecutor(root *Command) Executor

type Extra Uses

type Extra struct {
    // contains filtered or unexported fields

Extra is a set of tag information for a command

func (*Extra) GetValue Uses

func (e *Extra) GetValue(key interface{}) (interface{}, bool)

func (*Extra) SetValue Uses

func (e *Extra) SetValue(key, value interface{}) *Extra

type Flusher Uses

type Flusher interface {
    Flush() error

type Function Uses

type Function func(*Request, ResponseEmitter, Environment) error

Function is the type of function that Commands use. It reads from the Request, and writes results to the ResponseEmitter.

type HelpText Uses

type HelpText struct {
    // required
    Tagline               string            // used in <cmd usage>
    ShortDescription      string            // used in DESCRIPTION
    SynopsisOptionsValues map[string]string // mappings for synopsis generator

    // optional - whole section overrides
    Usage           string // overrides USAGE section
    LongDescription string // overrides DESCRIPTION section
    Options         string // overrides OPTIONS section
    Arguments       string // overrides ARGUMENTS section
    Subcommands     string // overrides SUBCOMMANDS section
    Synopsis        string // overrides SYNOPSIS field

HelpText is a set of strings used to generate command help text. The help text follows formats similar to man pages, but not exactly the same.

type MakeEnvironment Uses

type MakeEnvironment func(context.Context, *Request) (Environment, error)

MakeEnvironment takes a context and the request to construct the environment that is passed to the command's Run function. The user can define a function like this to pass it to cli.Run.

type MakeExecutor Uses

type MakeExecutor func(*Request, interface{}) (Executor, error)

MakeExecutor takes the request and environment variable to construct the executor that determines how to call the command - i.e. by calling Run or making an API request to a daemon. The user can define a function like this to pass it to cli.Run.

type MaybeError Uses

type MaybeError struct {
    Value interface{} // needs to be a pointer
    Error *Error
    // contains filtered or unexported fields

func (*MaybeError) Get Uses

func (m *MaybeError) Get() (interface{}, error)

func (*MaybeError) UnmarshalJSON Uses

func (m *MaybeError) UnmarshalJSON(data []byte) error

type OptMap Uses

type OptMap map[string]interface{}

type Option Uses

type Option interface {
    Name() string    // the main name of the option
    Names() []string // a list of unique names matched with user-provided flags

    Type() reflect.Kind  // value must be this type
    Description() string // a short string that describes this option

    WithDefault(interface{}) Option // sets the default value of the option
    Default() interface{}

    Parse(str string) (interface{}, error)

Option is used to specify a field that will be provided by a consumer

func BoolOption Uses

func BoolOption(names ...string) Option

func FloatOption Uses

func FloatOption(names ...string) Option

func Int64Option Uses

func Int64Option(names ...string) Option

func IntOption Uses

func IntOption(names ...string) Option

func NewOption Uses

func NewOption(kind reflect.Kind, names ...string) Option

constructor helper functions

func StringOption Uses

func StringOption(names ...string) Option

func StringsOption Uses

func StringsOption(names ...string) Option

func Uint64Option Uses

func Uint64Option(names ...string) Option

func UintOption Uses

func UintOption(names ...string) Option

type PostRunMap Uses

type PostRunMap map[PostRunType]func(Response, ResponseEmitter) error

PostRunMap is the map used in Command.PostRun.

type PostRunType Uses

type PostRunType string

PostRunType defines which PostRunFunc should be used

type ReqLog Uses

type ReqLog struct {
    Requests []*ReqLogEntry
    // contains filtered or unexported fields

ReqLog represents a request log.

func (*ReqLog) Add Uses

func (rl *ReqLog) Add(req *Request) *ReqLogEntry

Add ads an entry to the log for the given request.

func (*ReqLog) AddEntry Uses

func (rl *ReqLog) AddEntry(rle *ReqLogEntry)

AddEntry adds an entry to the log.

func (*ReqLog) ClearInactive Uses

func (rl *ReqLog) ClearInactive()

ClearInactive clears any inactive requests from the log.

func (*ReqLog) Finish Uses

func (rl *ReqLog) Finish(rle *ReqLogEntry)

func (*ReqLog) Report Uses

func (rl *ReqLog) Report() []*ReqLogEntry

Report generates a copy of all the entries in the requestlog

func (*ReqLog) SetKeepTime Uses

func (rl *ReqLog) SetKeepTime(t time.Duration)

type ReqLogEntry Uses

type ReqLogEntry struct {
    StartTime time.Time
    EndTime   time.Time
    Active    bool
    Command   string
    Options   map[string]interface{}
    Args      []string
    ID        int

ReqLogEntry represent a log entry for a request.

func (*ReqLogEntry) Copy Uses

func (r *ReqLogEntry) Copy() *ReqLogEntry

Copy copies a log entry and returns a pointer to the copy.

type Request Uses

type Request struct {
    Context       context.Context
    Root, Command *Command

    Path      []string
    Arguments []string
    Options   OptMap

    Files files.Directory
    // contains filtered or unexported fields

Request represents a call to a command from a consumer

func NewRequest Uses

func NewRequest(ctx context.Context,
    path []string,
    opts OptMap,
    args []string,
    file files.Directory,
    root *Command,
) (*Request, error)

NewRequest returns a request initialized with given arguments An non-nil error will be returned if the provided option values are invalid

func (*Request) BodyArgs Uses

func (req *Request) BodyArgs() StdinArguments

BodyArgs returns a scanner that returns arguments passed in the body as tokens.

Returns nil if there are no arguments to be consumed via stdin.

func (*Request) FillDefaults Uses

func (req *Request) FillDefaults() error

FillDefaults fills in default values if option has not been set.

func (*Request) ParseBodyArgs Uses

func (req *Request) ParseBodyArgs() error

ParseBodyArgs parses arguments in the request body.

func (*Request) SetOption Uses

func (req *Request) SetOption(name string, value interface{})

SetOption sets a request option.

type Response Uses

type Response interface {
    Request() *Request

    Error() *Error
    Length() uint64

    // Next returns the next emitted value.
    // The returned error can be a network or decoding error.
    Next() (interface{}, error)

Response is the result of a command request. Response is returned to the client.

func NewReaderResponse Uses

func NewReaderResponse(r io.Reader, req *Request) (Response, error)

NewReaderResponse creates a Response from the given reader.

type ResponseEmitter Uses

type ResponseEmitter interface {
    // Close closes the underlying transport.
    Close() error

    // CloseWithError closes the underlying transport and makes subsequent read
    // calls return the passed error.
    CloseWithError(error) error

    // SetLength sets the length of the output
    // err is an interface{} so we don't have to manually convert to error.
    SetLength(length uint64)

    // Emit sends a value.
    // If value is io.Reader we just copy that to the connection
    // other values are marshalled.
    Emit(value interface{}) error

ResponseEmitter encodes and sends the command code's output to the client. It is all a command can write to.

func NewFlushForwarder Uses

func NewFlushForwarder(re ResponseEmitter, f Flusher) ResponseEmitter

func NewWriterResponseEmitter Uses

func NewWriterResponseEmitter(w io.WriteCloser, req *Request) (ResponseEmitter, error)

NewWriterResponseEmitter creates a response emitter that sends responses to the given WriterCloser.

type Single Uses

type Single struct {
    Value interface{}

Single can be used to signal to any ResponseEmitter that only one value will be emitted. This is important e.g. for the http.ResponseEmitter so it can set the HTTP headers appropriately.

func (Single) GoString Uses

func (s Single) GoString() string

func (Single) String Uses

func (s Single) String() string

type StdinArguments Uses

type StdinArguments interface {

    // Scan reads in the next argument and returns true if there is an
    // argument to read.
    Scan() bool

    // Argument returns the next argument.
    Argument() string

    // Err returns any errors encountered when reading in arguments.
    Err() error

StdinArguments is used to iterate through arguments piped through stdin.

It closely mimics the bufio.Scanner interface but also implements the ReadCloser interface.

type TextEncoder Uses

type TextEncoder struct {
    // contains filtered or unexported fields

func (TextEncoder) Encode Uses

func (e TextEncoder) Encode(v interface{}) error



Package cmds imports 14 packages (graph) and is imported by 88 packages. Updated 2020-09-08. Refresh now. Tools for package owners.