logpipe

package
v1.23.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 30, 2024 License: Apache-2.0 Imports: 18 Imported by: 0

Documentation

Overview

Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.

Package logpipe implements reading csv logs from PostgreSQL logging_collector (https://www.postgresql.org/docs/current/runtime-config-logging.html) and convert them to JSON.

Index

Constants

View Source 
const FieldsPerRecord12 int = 23

FieldsPerRecord12 is the number of fields in a CSV log line in PostgreSQL 12 or below

View Source 
const FieldsPerRecord13 int = 24

FieldsPerRecord13 is the number of fields in a CSV log line since PostgreSQL 13

View Source 
const FieldsPerRecord14 int = 26

FieldsPerRecord14 is the number of fields in a CSV log line since PostgreSQL 14

View Source 
const LoggingCollectorRecordName = "postgres"

LoggingCollectorRecordName is the value of the logger field for logging_collector

View Source 
const PgAuditFieldsPerRecord int = 9

PgAuditFieldsPerRecord is the number of fields in a pgaudit log line

View Source 
const PgAuditRecordName = "pgaudit"

PgAuditRecordName is the value of the logger field for pgaudit

Variables

This section is empty.

Functions

This section is empty.

Types

type CSVReadWriter 

type CSVReadWriter interface {
	io.Writer
	Read() (record []string, err error)
}

CSVReadWriter is the interface for structs that are able to receive writes and parse CSV lines

type CSVRecordParser 

type CSVRecordParser interface {
	FromCSV(content []string) NamedRecord
	NamedRecord
}

CSVRecordParser is implemented by structs that can be filled when parsing a CSV line. The FromCSV method just stores the CSV record fields inside the struct fields. A validation check of the CSV fields should be performed by the caller. Also handling recover from panic should be provided by the caller, in order to take care of runtime error, e.g. index out of range because of CSV malformation

type CSVRecordReadWriter 

type CSVRecordReadWriter struct {
	io.Writer
	*csv.Reader
}

CSVRecordReadWriter wraps a csv.Reader and implements io.Writer. It parses CSV lines, that are then read through the csv.Reader.

func NewCSVRecordReadWriter 

func NewCSVRecordReadWriter(fieldsPerRecord int) *CSVRecordReadWriter

NewCSVRecordReadWriter returns a new CSVRecordReadWriter which parses CSV lines with an expected number of fields. It uses a single record for memory efficiency.

type ErrFieldCountExtended 

type ErrFieldCountExtended struct {
	Fields   []string
	Err      error
	Expected int
}

ErrFieldCountExtended is returned when the CSV line has an invalid number of fields

func LogFieldValidator 

func LogFieldValidator(fields int) *ErrFieldCountExtended

LogFieldValidator checks if the provided number of fields is valid or not for logging_collector logs

func (*ErrFieldCountExtended) Cause 

func (err *ErrFieldCountExtended) Cause() error

Cause returns the parent error

func (*ErrFieldCountExtended) Error 

func (err *ErrFieldCountExtended) Error() string

Error returns a description of the invalid record

type FieldsValidator 

type FieldsValidator func(int) *ErrFieldCountExtended

FieldsValidator is a function validating the number of fields for a specific log line to be parsed

type LineLogPipe 

type LineLogPipe struct {
	// contains filtered or unexported fields
}

LineLogPipe a pipe for a given format

func NewJSONLineLogPipe 

func NewJSONLineLogPipe(fileName string) *LineLogPipe

NewJSONLineLogPipe returns a logPipe for json format

func NewRawLineLogPipe 

func NewRawLineLogPipe(fileName, name string) *LineLogPipe

NewRawLineLogPipe returns a logPipe for raw output

func (*LineLogPipe) GetExecutedCondition 

func (p *LineLogPipe) GetExecutedCondition() *concurrency.Executed

GetExecutedCondition returns the condition that can be checked in order to be sure initialization has been done

func (*LineLogPipe) GetExitedCondition 

func (p *LineLogPipe) GetExitedCondition() *concurrency.Executed

GetExitedCondition returns the condition that can be checked in order to be sure initialization has been done

func (*LineLogPipe) Start 

func (p *LineLogPipe) Start(ctx context.Context) error

Start a new goroutine running the logging collector core, reading from a process logging raw strings to a file and redirecting its content to stdout in JSON format. The goroutine is started just once for a given file. All successive calls, that are referencing the same filename, will just check its existence

type LogPipe 

type LogPipe struct {
	// contains filtered or unexported fields
}

LogPipe creates a pipe for a given file

func NewLogPipe 

func NewLogPipe() *LogPipe

NewLogPipe returns a new LogPipe

func (*LogPipe) GetExitedCondition 

func (p *LogPipe) GetExitedCondition() *concurrency.Executed

GetExitedCondition returns the condition that can be checked in order to be sure the process has been executed

func (*LogPipe) GetInitializedCondition 

func (p *LogPipe) GetInitializedCondition() *concurrency.Executed

GetInitializedCondition returns the condition that can be checked in order to be sure initialization has been done

func (*LogPipe) Start 

func (p *LogPipe) Start(ctx context.Context) error

Start a new goroutine running the logging collector core, reading from a process logging in CSV to a file and redirecting its content to stdout in JSON format. The goroutine is started just once for a given file. All successive calls, that are referencing the same filename, will just check its existence

type LogRecordWriter 

type LogRecordWriter struct{}

LogRecordWriter implements the `RecordWriter` interface writing to the instance manager logger

func (*LogRecordWriter) Write 

func (writer *LogRecordWriter) Write(record NamedRecord)

Write writes the PostgreSQL log record to the instance manager logger

type LoggingRecord 

type LoggingRecord struct {
	LogTime              string `json:"log_time,omitempty"`
	Username             string `json:"user_name,omitempty"`
	DatabaseName         string `json:"database_name,omitempty"`
	ProcessID            string `json:"process_id,omitempty"`
	ConnectionFrom       string `json:"connection_from,omitempty"`
	SessionID            string `json:"session_id,omitempty"`
	SessionLineNum       string `json:"session_line_num,omitempty"`
	CommandTag           string `json:"command_tag,omitempty"`
	SessionStartTime     string `json:"session_start_time,omitempty"`
	VirtualTransactionID string `json:"virtual_transaction_id,omitempty"`
	TransactionID        string `json:"transaction_id,omitempty"`
	ErrorSeverity        string `json:"error_severity,omitempty"`
	SQLStateCode         string `json:"sql_state_code,omitempty"`
	Message              string `json:"message,omitempty"`
	Detail               string `json:"detail,omitempty"`
	Hint                 string `json:"hint,omitempty"`
	InternalQuery        string `json:"internal_query,omitempty"`
	InternalQueryPos     string `json:"internal_query_pos,omitempty"`
	Context              string `json:"context,omitempty"`
	Query                string `json:"query,omitempty"`
	QueryPos             string `json:"query_pos,omitempty"`
	Location             string `json:"location,omitempty"`
	ApplicationName      string `json:"application_name,omitempty"`
	BackendType          string `json:"backend_type,omitempty"`
	LeaderPid            string `json:"leader_pid,omitempty"`
	QueryID              string `json:"query_id,omitempty"`
}

LoggingRecord is used to store all the fields of the logging_collector CSV format

func (*LoggingRecord) FromCSV 

func (r *LoggingRecord) FromCSV(content []string) NamedRecord

FromCSV stores inside the record structure the relative fields of the CSV log record.

See https://www.postgresql.org/docs/current/runtime-config-logging.html section "19.8.4. Using CSV-Format Log Output".

func (*LoggingRecord) GetName 

func (r *LoggingRecord) GetName() string

GetName implements the NamedRecord interface

type NamedRecord 

type NamedRecord interface {
	GetName() string
}

NamedRecord is the interface for structs that have a name

type PgAuditLoggingDecorator 

type PgAuditLoggingDecorator struct {
	*LoggingRecord
	Audit         *PgAuditRecord `json:"audit,omitempty"`
	CSVReadWriter `json:"-"`
}

PgAuditLoggingDecorator stores all the fields of pgaudit CSV format

func NewPgAuditLoggingDecorator 

func NewPgAuditLoggingDecorator() *PgAuditLoggingDecorator

NewPgAuditLoggingDecorator builds PgAuditLoggingDecorator

func (*PgAuditLoggingDecorator) FromCSV 

func (r *PgAuditLoggingDecorator) FromCSV(content []string) NamedRecord

FromCSV implements the CSVRecordParser interface, parsing a LoggingRecord and then

func (*PgAuditLoggingDecorator) GetName 

func (r *PgAuditLoggingDecorator) GetName() string

GetName implements the NamedRecord interface

type PgAuditRecord 

type PgAuditRecord struct {
	AuditType      string `json:"audit_type,omitempty"`
	StatementID    string `json:"statement_id,omitempty"`
	SubstatementID string `json:"substatement_id,omitempty"`
	Class          string `json:"class,omitempty"`
	Command        string `json:"command,omitempty"`
	ObjectType     string `json:"object_type,omitempty"`
	ObjectName     string `json:"object_name,omitempty"`
	Statement      string `json:"statement,omitempty"`
	Parameter      string `json:"parameter,omitempty"`
}

PgAuditRecord stores all the fields of a pgaudit log line

type RecordWriter 

type RecordWriter interface {
	Write(r NamedRecord)
}

RecordWriter is the interface

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.