pxld

package module

v0.1.1 Latest Latest Go to latest Published: Apr 16, 2019 License: GPL-3.0 Imports: 8 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/tiket-oss/go-pxld

Links

Open Source Insights

README ¶

pxld

Decode your binary format ProxySQL query log.

Requirement

Go v1.11 or later

Build

go mod download
cd cmd/decoder
go build

Run

After you've build the decoder, just run the executable.

The File Format

5D 00 00 00 00 00 00 00 first 8 bytes is the length of a message, this one.
00 next byte denounce that this is a ProxySQL log message or not, if not 00, not valid.

The next after this line needs read_encoded_length function which itself needs mysql_decode_length function.

0A this is the thread id because it is less than or equal 0xFB, convert to uint64.
06 this is the length of username because it is less than or equal 0xFB, convert to uint64.
64 69 64 61 73 79 is the username in ASCII.
12 this is the length of schema because it is less than or equal 0xFB, convert to uint64.
69 6E 66 6F 72 6D 61 74 69 6F 6E 5F 73 63 68 65 6D 61 is the schema in ASCII.
0F this is the length of client address because it is less than or equal 0xFB, convert to uint64.
31 32 37 2E 30 2E 30 2E 31 3A 33 32 38 32 30 is the client address in ASCII.
FE this tell us to read the next 8 bytes as uint64.
FF FF FF FF FF FF FF FF this is the HID in uint64, if HID == UINT64_MAX then we don't have server address to read.
FE this tell us to read the next 8 bytes as uint64.
91 00 B3 4A 28 86 05 00 this is query start time in UNIX microseconds in uint64.
FE this tell us to read the next 8 bytes as uint64.
91 00 B3 4A 28 86 05 00 this is query end time in UNIX microseconds in uint64.
FE this tell us to read the next 8 bytes as uint64.
D6 1F BA 14 4D 1F 23 AE this is query digest in uint64, but it needs to be separated into two uint32 and then it can be printed into hex sprintf("0x%X%X", n1, n2) == 0x14BA1FD6AE231F4D.
0C this is the length of the actual query because it is less than or equal 0xFB, convert to uint64.
2E 30 2E 31 3A 33 32 38 32 30 00 A5 this the actual query in ASCII.

Then we can go to the next line and repeat.

Decoding Message Parts Length

To decode part length, first we must take the first byte of the part. Then we check if:

It is less or equal than 0xFB. If so we take this byte as the message length.
It is equal to 0xFC. If so, we take 2 bytes as the message length.
It is equal to 0xFD. If so, we take 3 bytes as the message length.
It is equal to 0xFE. If so, we take 8 bytes as the message length.

Documentation ¶

Index ¶

Constants
func GetClientAddr(dataStream io.Reader) (clientAddr string, err error)
func GetEncodedLength(dataStream io.Reader) (ln uint64, err error)
func GetEndAt(dataStream io.Reader) (endAt time.Time, err error)
func GetHID(dataStream io.Reader) (hid uint64, err error)
func GetMessage(messageLength uint64, dataStream io.Reader) (raw []byte, buf io.Reader, err error)
func GetMessageLength(dataStream io.Reader) (dataLength uint64, err error)
func GetQuery(dataStream io.Reader) (q string, err error)
func GetQueryDigest(dataStream io.Reader) (digest string, err error)
func GetSchema(dataStream io.Reader) (schema string, err error)
func GetServerAddr(dataStream io.Reader) (serverAddr string, err error)
func GetStartAt(dataStream io.Reader) (startAt time.Time, err error)
func GetString(dataStream io.Reader) (s string, err error)
func GetThreadID(dataStream io.Reader) (threadID uint64, err error)
func GetTime(dataStream io.Reader) (t time.Time, err error)
func GetUsername(dataStream io.Reader) (username string, err error)
func IsProxySQLQuery(dataStream io.Reader) (err error)
type LogLine
- func Decode(r io.Reader) (l []*LogLine, err error)
- func DecodeFile(fp string) (l []*LogLine, err error)
- func (l *LogLine) String() string

Constants ¶

View Source

const (
	ProxySQLQuery = 0
)

Variables ¶

This section is empty.

Functions ¶

func GetClientAddr ¶

func GetClientAddr(dataStream io.Reader) (clientAddr string, err error)

func GetEncodedLength ¶

func GetEncodedLength(dataStream io.Reader) (ln uint64, err error)

func GetEndAt ¶

func GetEndAt(dataStream io.Reader) (endAt time.Time, err error)

func GetHID ¶

func GetHID(dataStream io.Reader) (hid uint64, err error)

func GetMessage ¶

func GetMessage(messageLength uint64, dataStream io.Reader) (raw []byte, buf io.Reader, err error)

func GetMessageLength ¶

func GetMessageLength(dataStream io.Reader) (dataLength uint64, err error)

func GetQuery ¶

func GetQuery(dataStream io.Reader) (q string, err error)

func GetQueryDigest ¶

func GetQueryDigest(dataStream io.Reader) (digest string, err error)

func GetSchema ¶

func GetSchema(dataStream io.Reader) (schema string, err error)

func GetServerAddr ¶

func GetServerAddr(dataStream io.Reader) (serverAddr string, err error)

func GetStartAt ¶

func GetStartAt(dataStream io.Reader) (startAt time.Time, err error)

func GetString ¶

func GetString(dataStream io.Reader) (s string, err error)

func GetThreadID ¶

func GetThreadID(dataStream io.Reader) (threadID uint64, err error)

func GetTime ¶

func GetTime(dataStream io.Reader) (t time.Time, err error)

func GetUsername ¶

func GetUsername(dataStream io.Reader) (username string, err error)

func IsProxySQLQuery ¶

func IsProxySQLQuery(dataStream io.Reader) (err error)

Types ¶

type LogLine ¶

type LogLine struct {
	MessageLength uint64        `json:"message_length"`
	RawMessage    []byte        `json:"raw_message"` // this is without message length data prepended
	ThreadID      uint64        `json:"thread_id"`
	Username      string        `json:"username"`
	Schema        string        `json:"schema"`
	StartAt       time.Time     `json:"start_at"`
	EndAt         time.Time     `json:"end_at"`
	QueryDigest   string        `json:"query_digest"`
	HID           uint64        `json:"hid,omitempty"`
	ClientAddr    string        `json:"client_addr"`
	ServerAddr    string        `json:"server_addr,omitempty"` // this depends on HID value
	Query         string        `json:"query"`
	Duration      time.Duration `json:"duration_ns"`
}

func Decode ¶

func Decode(r io.Reader) (l []*LogLine, err error)

func DecodeFile ¶

func DecodeFile(fp string) (l []*LogLine, err error)

func (*LogLine) String ¶

func (l *LogLine) String() string

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
cmd
decoder

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL