Documentation
¶
Overview ¶
Package shquot contains various functions for quoting sequences of command arguments for literal interpretation by different shells and other similar intermediaries that process incoming command lines.
The functions all have the same signature, defined as type Q in this package, taking an array of arguments in the usual style passed to "execve" on a Unix/POSIX system.
While calling "execve" directly is always preferable to avoid misinterpretation by intermediaries, sometimes such preprocessing cannot be avoided. For example, remote command execution protocols like SSH often expect a single string to be interpreted by a shell.
Since each shell or intermediary has different details, it's important to select the correct quoting function for the target system or else the result may be misinterpreted.
The goal of functions in this package is to cause a command line to be interpreted as if it were passed to the "execve" C function, bypassing any special behaviors of intermediaries such as wildcard expansion, alias expansion, pipelines, etc. If any shell behaviors are accessible through crafted input to the corresponding function then that's always considered to be a bug unless specifically noted in the documentation for that function.
Example ¶
package main
import (
"fmt"
"github.com/apparentlymart/go-shquot/shquot"
)
func main() {
cmdline := []string{`echo`, `Hello, world!`}
fmt.Println("POSIXShell:", shquot.POSIXShell(cmdline))
fmt.Println("WindowsArgv:", shquot.WindowsArgv(cmdline))
fmt.Println("WindowsCmdExe+WindowsArgv:", shquot.WindowsCmdExe(shquot.WindowsArgv)(cmdline))
fmt.Println("Dockerfile:", shquot.Dockerfile(cmdline))
}
Output: POSIXShell: 'echo' 'Hello, world!' WindowsArgv: "echo" "Hello, world!" WindowsCmdExe+WindowsArgv: ^"echo^" ^"Hello, world^!^" Dockerfile: ["echo","Hello, world!"]
Index ¶
- func AlwaysValid(cmdline []string) bool
- func Dockerfile(cmdline []string) string
- func POSIXShell(cmdline []string) string
- func POSIXShellSplit(cmdline []string) (cmd, args string)
- func ValidUTF8(cmdline []string) bool
- func WindowsArgv(cmdline []string) string
- func WindowsArgvSplit(cmdline []string) (cmd, args string)
- func WindowsArgvValid(cmdline []string) bool
- type Q
- type QS
- type QV
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func AlwaysValid ¶
AlwaysValid is a placeholder implementation of QV that always returns true. This should be used only in callers that generalize over all Q and QV implementations to represent situations where the Q function has no unusual constraints.
func Dockerfile ¶
Dockerfile produces a string suitable for use as the argument to one of the directives RUN, CMD, and ENTRYPOINT in the Dockerfile format.
The "exec form" of all of these commands is just a JSON serialization of an array of strings, so this function is just a Q-compatible adapter to encoding/json.Marshal.
func POSIXShell ¶
POSIXShell quotes the given command line for interpretation by shells compatible with the POSIX shell standards, including most superset implementations like bash.
It will pass through individual arguments unchanged where possible for maximum readability. It will use backslash escapes for arguments that do not contain whitespace, and single quotes for arguments that do.
This function assumes a shell with the default value of the "IFS" variable, such that a single space will be interpreted as an argument separator.
The first argument is always quoted so that it will bypass alias expansion and function call behaviors in compliant shells.
func POSIXShellSplit ¶
POSIXShellSplit is a variant of POSIXShell that isolates the first argument (conventionally the program name) and returns it verbatim along with a quoted version of the remaining arguments.
func ValidUTF8 ¶
ValidUTF8 checks whether the elements of the given command line are all valid UTF-8 strings, returning false if not.
This is just a convenience wrapper for applying utf8.ValidString to each string in the slice.
func WindowsArgv ¶
WindowsArgv quotes arguments using conventions that lead to correct parsing by both the Windows API function CommandLineToArgvW and the parse_cmdline function in the Microsoft Visual C++ runtime library.
On Windows the final parsing of a command line string is the responsibility of the application itself, and so an application may employ any strategy it wishes to parse the command line.
In practice though, most command line Windows applications receive their arguments via the argv/argc parameters to function main, which contain the result of calling parse_cmdline.
Windows GUI applications receive a command line string as one argument to WinMain, but it has already been partially processed to remove the first argument that is conventionally the program name. Windows applications may instead choose to call GetCommandLine to obtain the full original string and then process it with the API function CommandLineToArgvW to obtain an argv/argc pair.
The parse_cmdline and CommandLineToArgvW implementations do not have identical behavior, but their behavior is compatible enough that this function can produce a string that can be processed successfully by both. For applications that use neither of these mechanisms there is no guarantee that any particular quoting scheme will work.
In particular note that programs that are not written in C or C++ (e.g. batch files, cmd scripts, Windows Script Host programs, .NET Framework applications, etc) are likely to have divergent processing rules that this function cannot guarantee to support.
There is one important caveat with this function: neither of the "argv" construction approaches supports escaping of quotes in the first argument, so quote characters cannot be reliably supported there. If any are found, this function will strip them out. In practice a quote character is never valid in a program name on Windows and so this should have no practical impact, but if you wish to detect that scenario you can first call WindowsArgvValid to determine whether a particular command line slice can be losslessly encoded by WindowsArgv.
func WindowsArgvSplit ¶
WindowsArgvSplit is a variant of WindowsArgv that quotes only the arguments in the given command line -- that is, indices 1 and greater in the given slice -- and just returns the command from index 0 verbatim to be quoted by another layer.
This is useful for Windows-style process-starting APIs where the command itself is isolated but the arguments are provided as a single, already-quoted string.
func WindowsArgvValid ¶
WindowsArgvValid is a helper for use alongside WindowsArgv to deal with the fact that the commonly-used Windows command line parsers do not support escaping of quotes in the first element of the command line.
This function returns false if the first element of the given cmdline contains quote characters, or true otherwise. If this function returns false then the return value of WindowsArgv for the same cmdline may be lossy.
Types ¶
type Q ¶
Q is the signature of all command line quoting functions in this package. This may be useful for calling applications that select dynamically which quoting mechanism to use and store a reference to the appropriate function to call later.
cmdline is a slice of string arguments where the first element is conventionally the command itself and any remaining elements are arguments to that command. This mimics the way command lines are passed to the execve function on a Unix (POSIX) system.
The strings in cmdline are assumed to be UTF-8 encoded. If not, the results of some functions may be incorrect.
func UnixTerminal ¶
UnixTerminal produces a quoting function that prepares a command line to pass through a Unix-style terminal driver.
On Unix systems we usually execute commands programmatically by directly executing them, rather than by delivering them to a shell through a terminal. However, in some rare cases a command line must be delivered as if typed manually at a terminal.
This function constructs a filter function that escapes control characters using the ^V control character, which is the default representation of the "literal next" (lnext) Terminal command. Since the terminal driver is only an intermediary on the way to some other shell, you must pass the quoting function for that final shell in order to obtain a function that applies both levels of quoting/escaping at once.
You don't need to use this function unless you are delivering a command line through a terminal driver. Such situations include programmatically typing commands into the console of a virtual machine via a virtualized keyboard, sending automated keystrokes to a real server via a serial console interface, or delivering a command to an interactive shell through a pseudo-terminal (pty). If you can avoid doing these things by directly executing the shell in a non-interactive mode, please do.
Note that the lnext command can potentially be remapped to a different control character using stty, in which case this function's result will be misinterpreted. Use this function only with terminals using the default ^V mapping.
The "lnext" command is a Minix extension and not part of the POSIX standard, but is in practice implemented on most modern Unix-style operating systems, including Linux. Additionally, most modern Unix shells accept interactive input in "raw" mode and thus any control characters are handled directly by the shell rather than by the terminal driver, and many treat lnext in the same way as the terminal driver would.
func ViaPowerShell ¶
ViaPowerShell wraps a "Split" quoting function to create a normal quoting function that produces a command string that can be passed to PowerShell to run the given command line as an external process.
PowerShell uses a single string for all of the arguments, which must itself be quoted in a manner suitable for the program being run. The wrapped split-quoting function provides the quoting for the arguments.
For example, if using PowerShell to run a program that uses the MSVC runtime's argument parsing functionality on Windows, use ViaPowerShell(WindowsArgvSplit) to get a suitably-quoted command line.
This function uses the PowerShell Start-Process cmdlet to force the command to be interpreted as an external program rather than as a cmdlet or other PowerShell command type.
Example ¶
package main
import (
"fmt"
"github.com/apparentlymart/go-shquot/shquot"
)
func main() {
cmdline := []string{`echo`, `Hello, world!`}
fmt.Println(shquot.ViaPowerShell(shquot.WindowsArgvSplit)(cmdline))
}
Output: & Start-Process -FilePath "echo" -ArgumentList "`"Hello, world!`""
func WindowsCmdExe ¶
WindowsCmdExe produces a quoting function that prepares a command line to pass through the Windows command interpreter cmd.exe.
Since cmd.exe is just an intermediary, the caller must provide another quoting function that deals with the subsequent layer of quoting. On Windows most of the command line processing is actually delegated to the application itself rather than the command interpreter, and so which wrapped quoting function to select depends on the target program. Most modern command line applications use the CommandLineToArgvW function for argument processing, and its escaping rules are implemented by WindowsArgv in this package.
Note that this extra level of quoting is necessary only for command lines that will pass through the command interpreter, such as generated command scripts. If you're calling the Windows CreateProcess API directly then you must not apply cmd.exe quoting, or the result will be incorrectly parsed.
This function cannot prevent expansion of Console Aliases, so the result is safe to run only in a command interpreter with no aliases configured.
type QS ¶
QS is a variant of Q that returns separate arguments for the unquoted command name (first element of cmdline, usually verbatim) and quoted remaining arguments, for use with intermediaries that require the command name to be provided out-of-band.
type QV ¶
QV is the signature of a function that checks if a given cmdline is valid. It returns true if the command line meets some validation constraint and false otherwise.
Most quoting functions can accept any command line consisting of valid UTF-8 strings, but some impose other constraints that may cause their result to be lossy, as described in each function's own documentation. In such cases, a separate function with a "Valid" suffix added allows a caller to check whether the given command line meets the constraints.
func AllValid ¶
AllValid combines multiple validation functions together to produce a single function that returns true only if all of the given checks return true. This is useful only for callers that generalize over all Q and QV implementations and that need to compose multiple checks in some cases.
Source Files