Documentation
¶
Overview ¶
The protoparse util package deals with reading and writing protocol buffer encoded records in, for example, files. functions provided here treat the underlying file as something that will be opened for either Reading or Writing (create or append)
Index ¶
- Constants
- func GetIP(a *pbcom.IPAddressWrapper) []byte
- func IPToRadixkey(b []byte, mask uint8) string
- func MarshalBytes(a *Footer) []byte
- func MaskStrToUint8(m string) (uint8, error)
- func OpenWithBufferSizes(p *RecordFile, readersize, writersize, openmode int) (err error, created bool)
- type FlatRecordFile
- type FootedRecordFile
- type Footer
- type Offset
- type PrefixTree
- type RecordFile
- func (p *RecordFile) Close() error
- func (p *RecordFile) Flush() (err error)
- func (p *RecordFile) Fname() string
- func (p *RecordFile) IncEntries(n uint64)
- func (p *RecordFile) OpenRead() error
- func (p *RecordFile) OpenWrite() error
- func (p *RecordFile) Read(b []byte) (int, error)
- func (p *RecordFile) Write(b []byte) (n int, err error)
- type RecordFiler
- type Section
Constants ¶
const ( RecordFile_Flat = iota RecordFile_FlatFooted RecordFile_Indexed )
const ( OMode_Read = iota OMode_Write )
Variables ¶
This section is empty.
Functions ¶
func GetIP ¶
func GetIP(a *pbcom.IPAddressWrapper) []byte
func IPToRadixkey ¶ added in v0.1.3
IPToRadixkey creates a binary string representation of an IP address. the length is 32 chars for IPv4 and 128 chars for IPv6. The mask is applied and zeroes out the bits it masks out on the resulting string.
func MarshalBytes ¶
MarshalBytes returns the bytes of [bytes of footer string][len of footer string][magic num] when this is writen to our underlying record file it is prepeneded by the length of the bytes mentioned above so this will appear as a normal record (but it won't parse normally)
func MaskStrToUint8 ¶
MaskStrToUint8 is a helper that just converts a possible mask string to a 10 based uint8.
func OpenWithBufferSizes ¶
func OpenWithBufferSizes(p *RecordFile, readersize, writersize, openmode int) (err error, created bool)
Opens the underlying reader with buffer sizes specified in the arguments. useful for larger tokens than the default 64k returns if it created a new file or if it just opened an existing one
Types ¶
type FlatRecordFile ¶
type FlatRecordFile struct {
*RecordFile
}
func NewFlatRecordFile ¶
func NewFlatRecordFile(fname string) *FlatRecordFile
func (*FlatRecordFile) Entries ¶
func (p *FlatRecordFile) Entries() (uint64, error)
Entries in a flat record file are not guaranteed to be correct. they are dependant to the position of the writer.
func (*FlatRecordFile) Footer ¶
func (p *FlatRecordFile) Footer() (*Footer, error)
Flat record files don't have headers.
func (*FlatRecordFile) Version ¶
func (p *FlatRecordFile) Version() uint16
type FootedRecordFile ¶
type FootedRecordFile struct {
*RecordFile
// contains filtered or unexported fields
}
func NewFootedRecordFile ¶
func NewFootedRecordFile(fname string) *FootedRecordFile
func (*FootedRecordFile) Close ¶
func (p *FootedRecordFile) Close() error
func (*FootedRecordFile) MakeFooter ¶
func (p *FootedRecordFile) MakeFooter() *Footer
func (*FootedRecordFile) OpenRead ¶
func (p *FootedRecordFile) OpenRead() error
OpenRead will try to read the footer
func (*FootedRecordFile) OpenWithFooter ¶
func (p *FootedRecordFile) OpenWithFooter(mode int) error
func (*FootedRecordFile) OpenWrite ¶
func (p *FootedRecordFile) OpenWrite() error
OpenWrite will try to read the footer
func (*FootedRecordFile) ReadFooter ¶
func (f *FootedRecordFile) ReadFooter() (*Footer, error)
Seeks to the end of the file, validates the magic number, reads the bytes of the footer, reads the footer as a string and calls ParseFooter on it. the last argument it returns are the number of bytes of the footer
type Footer ¶
type Footer struct {
}
A Footer is appended at the very end of a RecordFile (all types except the FlatRecordFile type ) The end of a footer should always be the magicbytes uint32 using that an application can easily see if a file is of our type Right before that the previous uint32 is the length of the footer bytes. The footer is a utf-8 encoded JSON string.In a sense it is a reversed entry than the length prefixed records it follows. The length should not include the 4 magic bytes. neither the 4 bytes of the footer length. it should be just the number of string bytes so in the end the size of the file should be the sum of all the entry bytes + footer size + 4. Length in the end of the file should be encoded in BigEndian
func ParseFooter ¶
type PrefixTree ¶
type PrefixTree struct {
// contains filtered or unexported fields
}
PrefixTree holds a radix tree which clients can insert IPs and masks in , and also lookup for their existence.
func NewPrefixTree ¶
func NewPrefixTree() PrefixTree
NewPrefixTree creates a new PrefixTree with an empty radix tree.
func (PrefixTree) Add ¶
func (pt PrefixTree) Add(IP net.IP, mask uint8)
Add adds an IP and a mask to that PrefixTree.
func (PrefixTree) ContainsIPMask ¶ added in v0.1.3
func (pt PrefixTree) ContainsIPMask(IP net.IP, mask uint8) bool
ContainsIPMask checks for the existance of that IP and mask in the PrefixTree. It performs a longest prefix match and if it is found it retuns true.
type RecordFile ¶
A record file knows the number of records it has stored, every record is preceded by a 32bit unsigned value that is the length of that record in Big Endian and after that the bytes of the record
func NewRecordFile ¶
func NewRecordFile(fname string) *RecordFile
func (*RecordFile) Flush ¶
func (p *RecordFile) Flush() (err error)
func (*RecordFile) Fname ¶
func (p *RecordFile) Fname() string
func (*RecordFile) IncEntries ¶
func (p *RecordFile) IncEntries(n uint64)
func (*RecordFile) OpenRead ¶
func (p *RecordFile) OpenRead() error
func (*RecordFile) OpenWrite ¶
func (p *RecordFile) OpenWrite() error
Source Files