cockroach: github.com/cockroachdb/cockroach/pkg/util/log Index | Files | Directories

package log

import "github.com/cockroachdb/cockroach/pkg/util/log"

Package log implements logging. There are three logging styles: named, V-style, events.

Named Functions

The functions Info, Warning, Error, and Fatal log their arguments at the severity level. All include formatting variants like Infof.

Examples:

log.Dev.Info(ctx, "prepare to repel boarders")
log.Ops.Info(ctx, "prepare to repel boarders")
log.Dev.Fatal(ctx, "initialization failed", err)
log.Dev.Infof(ctx, "client error: %s", err)

In these examples, the first word "Dev", "Ops" is the logging channel. Different channels are aimed at different audiences and can be redirected to different sinks.

If the channel name is omitted (e.g. log.Info), the Dev channel is used.

The second word e.g. "Info", "Fatal" etc is the severity level. For any given channel, users can filter out logging to a given severity level or higher.

V-Style

The V functions can be used to selectively enable logging at a call site. Invoking the binary with --vmodule=*=N will enable V functions at level N or higher. Invoking the binary with --vmodule="glob=N" will enable V functions at level N or higher with a filename matching glob.

Examples:

if log.V(2) {
	log.Info(ctx, "starting transaction...")
}

Additionally, severity functions also exist in a V variant for convenience. For example:

log.Ops.VWarningf(ctx, 2, "attention!")

aliases: if V(2) { log.Ops.Warningf(ctx, "attention!") }

Events

The Event functions log messages to an existing trace if one exists. The VEvent functions logs the message to a trace and also the Dev channel based on the V level.

Examples:

log.VEventf(ctx, 2, "client error; %s", err)

Output

Log output is buffered and written periodically using Flush. Programs should call Flush before exiting to guarantee all log output is written.

By default, all log statements write to files in a temporary directory. This package provides several flags that modify this behavior. These are provided via the util/log/logflags package; see InitFlags.

--logtostderr=LEVEL
  Logs are written to standard error as well as to files.
  Entries with severity below LEVEL are not written to stderr.
  "true" and "false" are also supported (everything / nothing).
--log-dir="..."
  Log files will be written to this directory by the main logger
  instead of the default target directory.
--log-file-verbosity=LEVEL
  Entries with severity below LEVEL are not written to the log file.
  "true" and "false" are also supported (everything / nothing).
--log-file-max-size=N
  Log files are rotated after reaching that size.
--log-group-max-size=N
  Log files are removed after the total size of all files generated
  by one logger reaches that size.

Other flags provide aids to debugging.

--vmodule=""
  The syntax of the argument is a comma-separated list of pattern=N,
  where pattern is a literal file name (minus the ".go" suffix) or
  "glob" pattern and N is a V level. For instance,
    --vmodule=gopher*=3
  sets the V level to 3 in all Go files whose names begin "gopher".

Index

Package Files

ambient_context.go channels.go clog.go doc.go event_log.go every_n.go exit_override.go file.go file_api.go file_log_gc.go file_sync_buffer.go flags.go format_crdb_v1.go format_json.go formats.go get_stacks.go intercept.go log.go log_bridge.go log_buffer.go log_channels_generated.go log_entry.go log_flush.go redact.go registry.go sinks.go stderr_redirect.go stderr_redirect_unix.go stderr_sink.go structured.go test_log_scope.go trace.go tracebacks.go vmodule.go

Constants

const FileNamePattern = `(?P<program>[^/.]+)\.(?P<host>[^/\.]+)\.` +
    `(?P<user>[^/\.]+)\.(?P<ts>[^/\.]+)\.(?P<pid>\d+)\.log`

FileNamePattern matches log files to avoid exposing non-log files accidentally and it splits the details of the filename into groups for easy parsing. The log file format is

{program}.{host}.{username}.{timestamp}.{pid}.log
cockroach.Brams-MacBook-Pro.bram.2015-06-09T16-10-48Z.30209.log

All underscore in process, host and username are escaped to double underscores and all periods are escaped to an underscore. For compatibility with Windows filenames, all colons from the timestamp (RFC3339) are converted from underscores (see FileTimePattern). Note this pattern is unanchored and becomes anchored through its use in LogFilePattern.

const FilePattern = "^(?:.*/)?" + FileNamePattern + "$"

FilePattern matches log file paths.

const FileTimeFormat = "2006-01-02T15_04_05Z07:00"

FileTimeFormat is RFC3339 with the colons replaced with underscores. It is the format used for timestamps in log file names. This removal of colons creates log files safe for Windows file systems.

const KeepRedactable = true

KeepRedactable can be used as an argument to SelectEditMode to indicate that the logs should retain their sensitive data markers so that they can be redacted later.

const MessageTimeFormat = "060102 15:04:05.999999"

MessageTimeFormat is the format of the timestamp in log message headers as used in time.Parse and time.Format.

Variables

var Dev loggerDev

Dev is a logger that logs to the DEV channel.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

var Health loggerHealth

Health is a logger that logs to the HEALTH channel.

The HEALTH channel is the channel used to report "background" operational events, initiated by CockroachDB or reporting on automatic processes:

- current resource usage, including critical resource usage. - node-node connection events, including connection errors and

gossip details.

- range and table leasing events. - up-, down-replication; range unavailability.

var MaybeSendCrashReport func(ctx context.Context, err error)

MaybeSendCrashReport is injected by package logcrash

var Ops loggerOps

Ops is a logger that logs to the OPS channel.

The OPS channel is the channel used to report "point" operational events, initiated by user operators or automation:

- operator or system actions on server processes: process starts,

stops, shutdowns, crashes (if they can be logged),
including each time: command-line parameters, current version being run.

- actions that impact the topology of a cluster: node additions,

removals, decommissions, etc.

- job-related initiation or termination. - cluster setting changes. - zone configuration changes.

var OrigStderr = func() *os.File {
    fd, err := dupFD(os.Stderr.Fd())
    if err != nil {
        panic(err)
    }

    return os.NewFile(fd, os.Stderr.Name())
}()

OrigStderr points to the original stderr stream when the process started. Note that it is generally not sound to restore os.Stderr and fd 2 from here as long as a stack of loggers (especially test log scopes) are active, as the loggers keep track of what they are redirecting themselves in a stack structure.

var Privileges loggerPrivileges

Privileges is a logger that logs to the PRIVILEGES channel.

The PRIVILEGES channel is the channel used to report data authorization changes, including:

- privilege grants/revocations on database, objects etc. - object ownership changes.

This is typically configured in "audit" mode, with event numbering and synchronous writes.

var Safe = redact.Safe

Safe constructs a SafeFormatter / SafeMessager. This is obsolete. Use redact.Safe directly. TODO(knz): Remove this.

var SensitiveAccess loggerSensitiveAccess

SensitiveAccess is a logger that logs to the SENSITIVE_ACCESS channel.

The SENSITIVE_ACCESS channel is the channel used to report SQL data access to sensitive data (when enabled):

- data access audit events (when table audit is enabled). - SQL statements executed by users with the ADMIN bit. - operations that write to `system` tables.

This is typically configured in "audit" mode, with event numbering and synchronous writes.

var Sessions loggerSessions

Sessions is a logger that logs to the SESSIONS channel.

The SESSIONS channel is the channel used to report client network activity:

- connections opened/closed. - authentication events: logins, failed attempts. - session and query cancellation.

This is typically configured in "audit" mode, with event numbering and synchronous writes.

var SqlExec loggerSqlExec

SqlExec is a logger that logs to the SQL_EXEC channel.

The SQL_EXEC channel is the channel used to report SQL execution on behalf of client connections:

- logical SQL statement executions (if enabled) - pgwire events (if enabled)

var SqlInternalPerf loggerSqlInternalPerf

SqlInternalPerf is a logger that logs to the SQL_INTERNAL_PERF channel.

The SQL_INTERNAL_PERF channel is like the SQL perf channel above but aimed at helping developers of CockroachDB itself. It exists as a separate channel so as to not pollute the SQL perf logging output with internal troubleshooting details.

var SqlPerf loggerSqlPerf

SqlPerf is a logger that logs to the SQL_PERF channel.

The SQL_PERF channel is the channel used to report SQL executions that are marked to be highlighted as "out of the ordinary" to facilitate performance investigations. This includes the "SQL slow query log".

Arguably, this channel overlaps with SQL_EXEC defined above. However, we keep them separate for backward-compatibility with previous versions, where the corresponding events were redirected to separate files.

var SqlSchema loggerSqlSchema

SqlSchema is a logger that logs to the SQL_SCHEMA channel.

The SQL_SCHEMA channel is the channel used to report changes to the SQL logical schema, excluding privilege and ownership changes (which are reported on the separate channel PRIVILEGES) and zone config changes (which go to OPS).

This includes:

- database/schema/table/sequence/view/type creation - adding/removing/changing table columns - changing sequence parameters

etc., more generally changes to the schema that affect the functional behavior of client apps using stored objects.

var Storage loggerStorage

Storage is a logger that logs to the STORAGE channel.

The STORAGE channel is the channel used to report low-level storage layer events (RocksDB/Pebble).

var UserAdmin loggerUserAdmin

UserAdmin is a logger that logs to the USER_ADMIN channel.

The USER_ADMIN channel is the channel used to report changes in users and roles, including:

- users added/dropped. - changes to authentication credentials, incl passwords, validity etc. - role grants/revocations. - role option grants/revocations.

This is typically configured in "audit" mode, with event numbering and synchronous writes.

func ApplyConfig Uses

func ApplyConfig(config logconfig.Config) (cleanupFn func(), err error)

ApplyConfig applies the given configuration.

The returned cleanup fn can be invoked by the caller to close asynchronous processes. NB: This is only useful in tests: for a long-running server process the cleanup function should likely not be called, to ensure that the file used to capture internal fd2 writes remains open up until the process entirely terminates. This ensures that any Go runtime assertion failures on the way to termination can be properly captured.

func DescribeAppliedConfig Uses

func DescribeAppliedConfig() string

DescribeAppliedConfig describes the current setup as effected by ApplyConfig(). This is useful in tests and also to check when something may be wrong with the logging configuration.

func DisableTracebacks Uses

func DisableTracebacks() func()

DisableTracebacks turns off tracebacks for log.Fatals. Returns a function that sets the traceback settings back to where they were. Only intended for use by tests.

func DumpStacks Uses

func DumpStacks(ctx context.Context)

DumpStacks produces a dump of the stack traces in the logging output.

func Error Uses

func Error(ctx context.Context, msg string)

Error logs to the DEV channel with severity ERROR. It extracts log tags from the context and logs them along with the given message.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The ERROR severity is used for situations that require special handling, when normal operation could not proceed as expected. Other operations can continue mostly unaffected.

func Errorf Uses

func Errorf(ctx context.Context, format string, args ...interface{})

Errorf logs to the DEV channel with severity ERROR, if logging has been enabled for the source file where the call is performed at the provided verbosity level, via the vmodule setting. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The ERROR severity is used for situations that require special handling, when normal operation could not proceed as expected. Other operations can continue mostly unaffected.

func ErrorfDepth Uses

func ErrorfDepth(ctx context.Context, depth int, format string, args ...interface{})

ErrorfDepth logs to the DEV channel with severity ERROR, offsetting the caller's stack frame by 'depth'. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The ERROR severity is used for situations that require special handling, when normal operation could not proceed as expected. Other operations can continue mostly unaffected.

func Event Uses

func Event(ctx context.Context, msg string)

Event looks for an opentracing.Trace in the context and logs the given message to it. If no Trace is found, it looks for an EventLog in the context and logs the message to it. If neither is found, does nothing.

func Eventf Uses

func Eventf(ctx context.Context, format string, args ...interface{})

Eventf looks for an opentracing.Trace in the context and formats and logs the given message to it. If no Trace is found, it looks for an EventLog in the context and logs the message to it. If neither is found, does nothing.

func ExpensiveLogEnabled Uses

func ExpensiveLogEnabled(ctx context.Context, level Level) bool

ExpensiveLogEnabled is used to test whether effort should be used to produce log messages whose construction has a measurable cost. It returns true if either the current context is recording the trace, or if the caller's verbosity is above level.

NOTE: This doesn't take into consideration whether tracing is generally enabled or whether a trace.EventLog or a trace.Trace (i.e. sp.netTr) is attached to ctx. In particular, if some OpenTracing collection is enabled (e.g. LightStep), that, by itself, does NOT cause the expensive messages to be enabled. "SET tracing" and friends, on the other hand, does cause these messages to be enabled, as it shows that a user has expressed particular interest in a trace.

Usage:

if ExpensiveLogEnabled(ctx, 2) {

msg := constructExpensiveMessage()
log.VEventf(ctx, 2, msg)

}

func Fatal Uses

func Fatal(ctx context.Context, msg string)

Fatal logs to the DEV channel with severity FATAL. It extracts log tags from the context and logs them along with the given message.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The FATAL severity is used for situations that require an immedate, hard server shutdown. A report is also sent to telemetry if telemetry is enabled.

func FatalChan Uses

func FatalChan() <-chan struct{}

FatalChan is closed when Fatal is called. This can be used to make the process stop handling requests while the final log messages and crash report are being written.

func FatalOnPanic Uses

func FatalOnPanic()

FatalOnPanic recovers from a panic and exits the process with a Fatal log. This is useful for avoiding a panic being caught through a CGo exported function or preventing HTTP handlers from recovering panics and ignoring them.

func Fatalf Uses

func Fatalf(ctx context.Context, format string, args ...interface{})

Fatalf logs to the DEV channel with severity FATAL, if logging has been enabled for the source file where the call is performed at the provided verbosity level, via the vmodule setting. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The FATAL severity is used for situations that require an immedate, hard server shutdown. A report is also sent to telemetry if telemetry is enabled.

func FatalfDepth Uses

func FatalfDepth(ctx context.Context, depth int, format string, args ...interface{})

FatalfDepth logs to the DEV channel with severity FATAL, offsetting the caller's stack frame by 'depth'. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The FATAL severity is used for situations that require an immedate, hard server shutdown. A report is also sent to telemetry if telemetry is enabled.

func FetchEntriesFromFiles Uses

func FetchEntriesFromFiles(
    startTimestamp, endTimestamp int64,
    maxEntries int,
    pattern *regexp.Regexp,
    editMode EditSensitiveData,
) ([]logpb.Entry, error)

FetchEntriesFromFiles fetches all available log entries on disk that are between the 'startTimestamp' and 'endTimestamp'. It will stop reading new files if the number of entries exceeds 'maxEntries'. Log entries are further filtered by the regexp 'pattern' if provided. The logs entries are returned in reverse chronological order.

func FinishEventLog Uses

func FinishEventLog(ctx context.Context)

FinishEventLog closes the event log in the context (see WithEventLog). Concurrent and subsequent calls to record events are allowed.

func Flush Uses

func Flush()

Flush explicitly flushes all pending log file I/O. See also flushDaemon() that manages background (asynchronous) flushes, and signalFlusher() that manages flushes in reaction to a user signal.

func FormatLegacyEntry Uses

func FormatLegacyEntry(e logpb.Entry, w io.Writer) error

FormatLegacyEntry writes the legacy log entry to the specified writer.

func FormatWithContextTags Uses

func FormatWithContextTags(ctx context.Context, format string, args ...interface{}) string

FormatWithContextTags formats the string and prepends the context tags.

Redaction markers are *not* inserted. The resulting string is generally unsafe for reporting.

func GetFormatterDocs Uses

func GetFormatterDocs() map[string]string

GetFormatterDocs returns the embedded documentation for all the supported formats.

func GetLogReader Uses

func GetLogReader(filename string, restricted bool) (io.ReadCloser, error)

GetLogReader returns a reader for the specified filename. In restricted mode, the filename must be the base name of a file in this process's log directory (this is safe for cases when the filename comes from external sources, such as the admin UI via HTTP). In unrestricted mode any path is allowed, relative to the current directory, with the added feature that simple (base name) file names will be searched in this process's log directory if not found in the current directory.

TODO(knz): make this work for secondary loggers too.

func HasSpanOrEvent Uses

func HasSpanOrEvent(ctx context.Context) bool

HasSpanOrEvent returns true if the context has a span or event that should be logged to.

func Info Uses

func Info(ctx context.Context, msg string)

Info logs to the DEV channel with severity INFO. It extracts log tags from the context and logs them along with the given message.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The INFO severity is used for informational messages, when no action is required as a result.

func Infof Uses

func Infof(ctx context.Context, format string, args ...interface{})

Infof logs to the DEV channel with severity INFO, if logging has been enabled for the source file where the call is performed at the provided verbosity level, via the vmodule setting. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The INFO severity is used for informational messages, when no action is required as a result.

func InfofDepth Uses

func InfofDepth(ctx context.Context, depth int, format string, args ...interface{})

InfofDepth logs to the DEV channel with severity INFO, offsetting the caller's stack frame by 'depth'. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The INFO severity is used for informational messages, when no action is required as a result.

func Intercept Uses

func Intercept(ctx context.Context, f InterceptorFn)

Intercept diverts log traffic to the given function `f`. When `f` is not nil, the logging package begins operating at full verbosity (i.e. `V(n) == true` for all `n`) but nothing will be printed to the logs. Instead, `f` is invoked for each log entry.

To end log interception, invoke `Intercept()` with `f == nil`. Note that interception does not terminate atomically, that is, the originally supplied callback may still be invoked after a call to `Intercept` with `f == nil`.

func IsActive Uses

func IsActive() (active bool, firstUse string)

IsActive returns true iff the main logger already has some events logged, or some secondary logger was created with configuration taken from the main logger.

This is used to assert that configuration is performed before logging has been used for the first time.

func ListLogFiles Uses

func ListLogFiles() (logFiles []logpb.FileInfo, err error)

ListLogFiles returns a slice of logpb.FileInfo structs for each log file on the local node, in any of the configured log directories.

func LoggingToStderr Uses

func LoggingToStderr(s Severity) bool

LoggingToStderr returns true if log messages of the given severity sent to the channel's logger are also visible on the process' external stderr. This is used e.g. by the startup code to announce server details both on the external stderr and to the log file.

This is also the logic used by Shout calls.

func MakeFileInfo Uses

func MakeFileInfo(details logpb.FileDetails, info os.FileInfo) logpb.FileInfo

MakeFileInfo constructs a FileInfo from FileDetails and os.FileInfo.

func MakeLegacyEntry Uses

func MakeLegacyEntry(
    ctx context.Context,
    s Severity,
    c Channel,
    depth int,
    redactable bool,
    format string,
    args ...interface{},
) (res logpb.Entry)

MakeLegacyEntry creates an logpb.Entry.

func NewStdLogger Uses

func NewStdLogger(severity Severity, prefix string) *stdLog.Logger

NewStdLogger creates a *stdLog.Logger that forwards messages to the CockroachDB logs with the specified severity.

The prefix should be the path of the package for which this logger is used. The prefix will be concatenated directly with the name of the file that triggered the logging.

func ParseLogFilename Uses

func ParseLogFilename(filename string) (logpb.FileDetails, error)

ParseLogFilename parses a filename into FileDetails if it matches the pattern for log files. If the filename does not match the log file pattern, an error is returned.

func ResetExitFunc Uses

func ResetExitFunc()

ResetExitFunc undoes any prior call to SetExitFunc.

func SetExitFunc Uses

func SetExitFunc(hideStack bool, f func(exit.Code))

SetExitFunc allows setting a function that will be called to exit the process when a Fatal message is generated. The supplied bool, if true, suppresses the stack trace, which is useful for test callers wishing to keep the logs reasonably clean.

Use ResetExitFunc() to reset.

func SetNodeIDs Uses

func SetNodeIDs(clusterID string, nodeID int32)

SetNodeIDs stores the Node and Cluster ID for further reference.

func SetTenantIDs Uses

func SetTenantIDs(tenantID string, sqlInstanceID int32)

SetTenantIDs stores the tenant ID and instance ID for further reference.

func SetVModule Uses

func SetVModule(value string) error

SetVModule alters the vmodule logging level to the passed in value.

func Shout Uses

func Shout(ctx context.Context, sev Severity, msg string)

Shout logs to channel DEV, and also to the real stderr if logging is currently redirected to a file.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

func Shoutf Uses

func Shoutf(ctx context.Context, sev Severity, format string, args ...interface{})

Shoutf logs to channel DEV, and also to the real stderr if logging is currently redirected to a file. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

func StartAlwaysFlush Uses

func StartAlwaysFlush()

StartAlwaysFlush configures all loggers to start flushing writes. This is used e.g. in `cockroach start` when an error occurs, to ensure that all log writes from the point the error occurs are flushed to logs (in case the error degenerates into a panic / segfault on the way out).

func StructuredEvent Uses

func StructuredEvent(ctx context.Context, event eventpb.EventPayload)

StructuredEvent emits a structured event to the debug log.

func TemporarilyDisableFileGCForMainLogger Uses

func TemporarilyDisableFileGCForMainLogger() (cleanup func())

TemporarilyDisableFileGCForMainLogger disables the file-based GC until the cleanup fn is called. Note that the behavior is undefined if this is called from multiple concurrent goroutines.

For background about why this function exists, see: https://github.com/cockroachdb/cockroach/issues/36861#issuecomment-483589446

func TestingClearServerIdentifiers Uses

func TestingClearServerIdentifiers()

TestingClearServerIdentifiers clears the server identity from the logging system. This is for use in tests that start multiple servers with conflicting identities subsequently. See discussion here: https://github.com/cockroachdb/cockroach/issues/58938

func TestingResetActive Uses

func TestingResetActive()

TestingResetActive clears the active bit. This is for use in tests that use stderr redirection alongside other tests that use logging.

func TestingSetRedactable Uses

func TestingSetRedactable(redactableLogs bool) (cleanup func())

TestingSetRedactable sets the redactable flag on the file output of the debug logger for usage in a test. The caller is responsible for calling the cleanup function. This is exported for use in tests only -- it causes the logging configuration to be at risk of leaking unsafe information due to asynchronous direct writes to fd 2 / os.Stderr.

See the discussion on SetupRedactionAndStderrRedirects() for details.

This is not safe for concurrent use with logging operations.

func V Uses

func V(level Level) bool

V returns true if the logging verbosity is set to the specified level or higher.

See also ExpensiveLogEnabled().

TODO(andrei): Audit uses of V() and see which ones should actually use the newer ExpensiveLogEnabled().

func VDepth Uses

func VDepth(l Level, depth int) bool

VDepth reports whether verbosity at the call site is at least the requested level.

func VErrEvent Uses

func VErrEvent(ctx context.Context, level Level, msg string)

VErrEvent either logs an error message to the DEV channel (which also outputs to the active trace or event log) or to the trace/event log alone, depending on whether the specified verbosity level is active.

func VErrEventf Uses

func VErrEventf(ctx context.Context, level Level, format string, args ...interface{})

VErrEventf either logs an error message to the DEV Channel (which also outputs to the active trace or event log) or to the trace/event log alone, depending on whether the specified verbosity level is active.

func VErrEventfDepth Uses

func VErrEventfDepth(
    ctx context.Context, depth int, level Level, format string, args ...interface{},
)

VErrEventfDepth performs the same as VErrEventf but checks the verbosity level at the given depth in the call stack.

func VErrorf Uses

func VErrorf(ctx context.Context, level Level, format string, args ...interface{})

VErrorf logs to the DEV channel with severity ERROR. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The ERROR severity is used for situations that require special handling, when normal operation could not proceed as expected. Other operations can continue mostly unaffected.

func VEvent Uses

func VEvent(ctx context.Context, level Level, msg string)

VEvent either logs a message to the DEV channel (which also outputs to the active trace or event log) or to the trace/event log alone, depending on whether the specified verbosity level is active.

func VEventf Uses

func VEventf(ctx context.Context, level Level, format string, args ...interface{})

VEventf either logs a message to the DEV channel (which also outputs to the active trace or event log) or to the trace/event log alone, depending on whether the specified verbosity level is active.

func VEventfDepth Uses

func VEventfDepth(ctx context.Context, depth int, level Level, format string, args ...interface{})

VEventfDepth performs the same as VEventf but checks the verbosity level at the given depth in the call stack.

func VFatalf Uses

func VFatalf(ctx context.Context, level Level, format string, args ...interface{})

VFatalf logs to the DEV channel with severity FATAL. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The FATAL severity is used for situations that require an immedate, hard server shutdown. A report is also sent to telemetry if telemetry is enabled.

func VInfof Uses

func VInfof(ctx context.Context, level Level, format string, args ...interface{})

VInfof logs to the DEV channel with severity INFO. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The INFO severity is used for informational messages, when no action is required as a result.

func VWarningf Uses

func VWarningf(ctx context.Context, level Level, format string, args ...interface{})

VWarningf logs to the DEV channel with severity WARNING. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The WARNING severity is used for situations which may require special handling, while normal operation is expected to resume automatically.

func Warning Uses

func Warning(ctx context.Context, msg string)

Warning logs to the DEV channel with severity WARNING. It extracts log tags from the context and logs them along with the given message.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The WARNING severity is used for situations which may require special handling, while normal operation is expected to resume automatically.

func Warningf Uses

func Warningf(ctx context.Context, format string, args ...interface{})

Warningf logs to the DEV channel with severity WARNING, if logging has been enabled for the source file where the call is performed at the provided verbosity level, via the vmodule setting. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The WARNING severity is used for situations which may require special handling, while normal operation is expected to resume automatically.

func WarningfDepth Uses

func WarningfDepth(ctx context.Context, depth int, format string, args ...interface{})

WarningfDepth logs to the DEV channel with severity WARNING, offsetting the caller's stack frame by 'depth'. It extracts log tags from the context and logs them along with the given message. Arguments are handled in the manner of fmt.Printf.

The DEV channel is the channel used during development, to collect log details useful for troubleshooting when it is unclear which other channel to use. It is also the default logging channel in CockroachDB, when the caller does not indicate a channel.

This channel is special in that there are no constraints as to what may or may not be logged on it. Conversely, users in production deployments are invited to not collect DEV logs in centralized logging facilities, because they likely contain sensitive operational data.

The WARNING severity is used for situations which may require special handling, while normal operation is expected to resume automatically.

func WithEventLog Uses

func WithEventLog(ctx context.Context, family, title string) context.Context

WithEventLog creates and embeds a trace.EventLog in the context, causing future logging and event calls to go to the EventLog. The current context must not have an existing open span.

func WithNoEventLog Uses

func WithNoEventLog(ctx context.Context) context.Context

WithNoEventLog creates a context which no longer has an embedded event log.

type AmbientContext Uses

type AmbientContext struct {
    // Tracer is used to open spans (see AnnotateCtxWithSpan).
    Tracer *tracing.Tracer
    // contains filtered or unexported fields
}

AmbientContext is a helper type used to "annotate" context.Contexts with log tags and a Tracer or EventLog. It is intended to be embedded into various server components.

Example:

type SomeServer struct {
  log.AmbientContext
  ...
}

ac := AmbientContext{Tracer: tracing.NewTracer()}
ac.AddLogTag("n", 1)

s := &SomeServer{
  AmbientContext: ac
  ...
}

// on an operation with context ctx
ctx = s.AnnotateCtx(ctx)
...

// run a worker
s.stopper.RunWorker(func() {
  ctx := s.AnnotateCtx(context.Background())
  ...
})

// start a background operation
ctx, span := s.AnnotateCtxWithSpan(context.Background(), "some-op")
defer span.Finish()
...

func (*AmbientContext) AddLogTag Uses

func (ac *AmbientContext) AddLogTag(name string, value interface{})

AddLogTag adds a tag to the ambient context.

func (*AmbientContext) AnnotateCtx Uses

func (ac *AmbientContext) AnnotateCtx(ctx context.Context) context.Context

AnnotateCtx annotates a given context with the information in AmbientContext:

- the EventLog is embedded in the context if the context doesn't already
  have an event log or an open trace.
- the log tags in AmbientContext are added (if ctx doesn't already have
them). If the tags already exist, the values from the AmbientContext
overwrite the existing values, but the order of the tags might change.

For background operations, context.Background() should be passed; however, in that case it is strongly recommended to open a span if possible (using AnnotateCtxWithSpan).

func (*AmbientContext) AnnotateCtxWithSpan Uses

func (ac *AmbientContext) AnnotateCtxWithSpan(
    ctx context.Context, opName string,
) (context.Context, *tracing.Span)

AnnotateCtxWithSpan annotates the given context with the information in AmbientContext (see AnnotateCtx) and opens a span.

If the given context has a span, the new span is a child of that span. Otherwise, the Tracer in AmbientContext is used to create a new root span.

The caller is responsible for closing the span (via Span.Finish).

func (*AmbientContext) FinishEventLog Uses

func (ac *AmbientContext) FinishEventLog()

FinishEventLog closes the event log. Concurrent and subsequent calls to record events from contexts that use this event log embedded are allowed.

func (*AmbientContext) ResetAndAnnotateCtx Uses

func (ac *AmbientContext) ResetAndAnnotateCtx(ctx context.Context) context.Context

ResetAndAnnotateCtx annotates a given context with the information in AmbientContext, but unlike AnnotateCtx, it drops all log tags in the supplied context before adding the ones from the AmbientContext.

func (*AmbientContext) SetEventLog Uses

func (ac *AmbientContext) SetEventLog(family, title string)

SetEventLog sets up an event log. Annotated contexts log into this event log (unless there's an open Span).

type Channel Uses

type Channel = logpb.Channel

Channel aliases a type.

type ChannelLogger Uses

type ChannelLogger interface {
    // Infof logs to the channel with severity INFO.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    Infof(ctx context.Context, format string, args ...interface{})

    // VInfof logs to the channel with severity INFO,
    // if logging has been enabled for the source file where the call is
    // performed at the provided verbosity level, via the vmodule setting.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    VInfof(ctx context.Context, level Level, format string, args ...interface{})

    // Info logs to the channel with severity INFO.
    // It extracts log tags from the context and logs them along with the given
    // message.
    Info(ctx context.Context, msg string)

    // InfofDepth logs to the channel with severity INFO,
    // offsetting the caller's stack frame by 'depth'.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    InfofDepth(ctx context.Context, depth int, format string, args ...interface{})

    // Warningf logs to the channel with severity WARNING.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    Warningf(ctx context.Context, format string, args ...interface{})

    // VWarningf logs to the channel with severity WARNING,
    // if logging has been enabled for the source file where the call is
    // performed at the provided verbosity level, via the vmodule setting.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    VWarningf(ctx context.Context, level Level, format string, args ...interface{})

    // Warning logs to the channel with severity WARNING.
    // It extracts log tags from the context and logs them along with the given
    // message.
    Warning(ctx context.Context, msg string)

    // WarningfDepth logs to the channel with severity WARNING,
    // offsetting the caller's stack frame by 'depth'.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    WarningfDepth(ctx context.Context, depth int, format string, args ...interface{})

    // Errorf logs to the channel with severity ERROR.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    Errorf(ctx context.Context, format string, args ...interface{})

    // VErrorf logs to the channel with severity ERROR,
    // if logging has been enabled for the source file where the call is
    // performed at the provided verbosity level, via the vmodule setting.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    VErrorf(ctx context.Context, level Level, format string, args ...interface{})

    // Error logs to the channel with severity ERROR.
    // It extracts log tags from the context and logs them along with the given
    // message.
    Error(ctx context.Context, msg string)

    // ErrorfDepth logs to the channel with severity ERROR,
    // offsetting the caller's stack frame by 'depth'.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    ErrorfDepth(ctx context.Context, depth int, format string, args ...interface{})

    // Fatalf logs to the channel with severity FATAL.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    Fatalf(ctx context.Context, format string, args ...interface{})

    // VFatalf logs to the channel with severity FATAL,
    // if logging has been enabled for the source file where the call is
    // performed at the provided verbosity level, via the vmodule setting.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    VFatalf(ctx context.Context, level Level, format string, args ...interface{})

    // Fatal logs to the channel with severity FATAL.
    // It extracts log tags from the context and logs them along with the given
    // message.
    Fatal(ctx context.Context, msg string)

    // FatalfDepth logs to the channel with severity FATAL,
    // offsetting the caller's stack frame by 'depth'.
    // It extracts log tags from the context and logs them along with the given
    // message. Arguments are handled in the manner of fmt.Printf.
    FatalfDepth(ctx context.Context, depth int, format string, args ...interface{})

    // Shout logs to the channel, and also to the real stderr if logging
    // is currently redirected to a file.
    Shout(ctx context.Context, sev Severity, msg string)

    // Shoutf logs to the channel, and also to the real stderr if
    // logging is currently redirected to a file. Arguments are handled in
    // the manner of fmt.Printf.
    Shoutf(ctx context.Context, sev Severity, format string, args ...interface{})
}

ChannelLogger is a helper interface to ease the run-time selection of channels. We do not force use of ChannelLogger when instantiating the logger objects below (e.g. by giving them the interface type), to ensure the calls remain inlinable in the common case.

Note that casting a channel logger to the interface type yields a heap allocation: it may be useful for performance to pre-allocate interface references in the global scope.

type EditSensitiveData Uses

type EditSensitiveData int

EditSensitiveData describes how the messages in log entries should be edited through the API.

const (

    // WithFlattenedSensitiveData is the log including sensitive data,
    // but markers stripped.
    WithFlattenedSensitiveData EditSensitiveData = confValid
    // WithMarkedSensitiveData is the "raw" log with sensitive data markers included.
    WithMarkedSensitiveData EditSensitiveData = confValid | withKeepMarkers
    // WithoutSensitiveDataNorMarkers is the log with the sensitive data
    // redacted, and markers stripped.
    WithoutSensitiveDataNorMarkers EditSensitiveData = confValid | withRedaction
    // WithoutSensitiveData is the log with the sensitive data redacted,
    // but markers included.
    WithoutSensitiveData EditSensitiveData = confValid | withKeepMarkers | withRedaction
)

func SelectEditMode Uses

func SelectEditMode(redact, keepRedactable bool) EditSensitiveData

SelectEditMode returns an EditSensitiveData value that's suitable for use with NewDecoder depending on client-side desired "redact" and "keep redactable" flags. (See the documentation for the Logs and LogFile RPCs and that of the 'merge-logs' CLI command.)

type EntryDecoder Uses

type EntryDecoder struct {
    // contains filtered or unexported fields
}

EntryDecoder reads successive encoded log entries from the input buffer. Each entry is preceded by a single big-ending uint32 describing the next entry's length.

func NewEntryDecoder Uses

func NewEntryDecoder(in io.Reader, editMode EditSensitiveData) *EntryDecoder

NewEntryDecoder creates a new instance of EntryDecoder.

func (*EntryDecoder) Decode Uses

func (d *EntryDecoder) Decode(entry *logpb.Entry) error

Decode decodes the next log entry into the provided protobuf message.

type EveryN Uses

type EveryN struct {
    util.EveryN
}

EveryN provides a way to rate limit spammy log messages. It tracks how recently a given log message has been emitted so that it can determine whether it's worth logging again.

func Every Uses

func Every(n time.Duration) EveryN

Every is a convenience constructor for an EveryN object that allows a log message every n duration.

func (*EveryN) ShouldLog Uses

func (e *EveryN) ShouldLog() bool

ShouldLog returns whether it's been more than N time since the last event.

type InterceptorFn Uses

type InterceptorFn func(entry logpb.Entry)

InterceptorFn is the type of function accepted by Intercept().

type Level Uses

type Level int32

Level specifies a level of verbosity for V logs. *Level implements flag.Value; the --verbosity flag is of type Level and should be modified only through the flag.Value interface.

Level is exported because it appears in the arguments to V and is the type of the v flag, which can be set programmatically. It's a distinct type because we want to discriminate it from logType. Variables of type level are only changed under loggerT.mu. The --verbosity flag is read only with atomic ops, so the state of the logging module is consistent.

Level is treated as a sync/atomic int32.

func (*Level) Set Uses

func (l *Level) Set(value string) error

Set is part of the flag.Value interface.

func (*Level) String Uses

func (l *Level) String() string

String is part of the flag.Value interface.

type Severity Uses

type Severity = logpb.Severity

Severity aliases a type.

type TestLogScope Uses

type TestLogScope struct {
    // contains filtered or unexported fields
}

TestLogScope represents the lifetime of a logging output. It ensures that the log files are stored in a directory specific to a test, and enforces that logging output is not written to this directory beyond the lifetime of the scope.

func Scope Uses

func Scope(t tShim) *TestLogScope

Scope creates a TestLogScope which corresponds to the lifetime of a temporary logging directory. If -show-logs was passed on the command line, this is a no-op. Otherwise, it behaves like ScopeWithoutShowLogs().

See the documentation of ScopeWithoutShowLogs() for API usage and restrictions.

func ScopeWithoutShowLogs Uses

func ScopeWithoutShowLogs(t tShim) (sc *TestLogScope)

ScopeWithoutShowLogs creates a TestLogScope which corresponds to the lifetime of a temporary logging directory. The logging directory is named after the calling test. ScopeWithoutShowLogs ignores the -show-logs flag and should be used for tests that require the logs go to files in any case.

The TestLogScope redirects any log calls "under it" to the temporary directory. When the scope ends, the previous configuration is restored.

ScopeWithoutShowLogs() is only valid if logging was not yet active. If it was, the test fails with an assertion error. The motivation for this restriction is to simplify reasoning by users of this facility: if a user has set up their code so that logging goes to files already, they are signaling that they want logging to go there and not elsewhere. In that case, it is undesirable to come along with a new random directory and take logging over there.

ScopeWithoutShowLogs() does not enable redirection of internal stderr writes to files. Tests that wish to use that facility should call the other APIs in these package after setting up a TestLogScope.

func (*TestLogScope) Close Uses

func (l *TestLogScope) Close(t tShim)

Close cleans up a TestLogScope. The directory and its contents are deleted, unless the test has failed and the directory is non-empty.

func (*TestLogScope) GetDirectory Uses

func (l *TestLogScope) GetDirectory() string

GetDirectory retrieves the log directory for this scope.

func (*TestLogScope) Rotate Uses

func (l *TestLogScope) Rotate(t tShim)

Rotate closes the current log files so that the next log call will reopen them with current settings. This is useful when e.g. a test changes the logging configuration after opening a test log scope.

Directories

PathSynopsis
channel
eventpbPackage eventpb defines standardized payloads for cluster-level and SQL-level event reporting.
logconfigPackage logconfig manages the configuration of the logging channels and their logging sinks.
logcrash
logflags
logpb
severity

Package log imports 48 packages (graph) and is imported by 2010 packages. Updated 2021-01-21. Refresh now. Tools for package owners.