Documentation
¶
Index ¶
- func Compress(output io.Writer) (*pgzip.Writer, func(), error)
- func Concatenate(inputs []*os.File, output io.Writer) error
- func ConcatenateAndCompress(inputs []*os.File, output *os.File) error
- func DgutFiles(inputs []string, outputDir string) error
- func GroupFiles(inputs []*os.File, output *os.File) error
- func LogFiles(inputs []*os.File, output *os.File) error
- func Merge(inputs []*os.File, output io.Writer, streamFunc Merger) error
- func MergeAndCompress(inputs []*os.File, output *os.File, streamFunc Merger) error
- func MergeSortedFiles(inputs []string) (io.ReadCloser, func() error, error)
- func MergeSummaryLines(data io.ReadCloser, matchColumns, summaryColumns int, ...) error
- func StatFiles(inputs []*os.File, output *os.File) error
- func UserGroupFiles(inputs []*os.File, output *os.File) error
- type MatchingSummaryLineMerger
- type Merger
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Compress ¶
Compress takes an io writer as its input, and compresses it. It returns this, along with a function to close the writer and an error status.
func Concatenate ¶
Concatenate takes a list of open files as its input, and an io.Writer as its output. It concatenates the contents of the inputs into the output.
func ConcatenateAndCompress ¶
ConcatenateAndCompress takes a list of open files as its input, and an open file for its output. It writes to the output the compressed, concatenated inputs.
func DgutFiles ¶
DgutFiles merges the pre-sorted dgut files, summing consecutive lines with the same first 4 columns, and outputs the results to an embedded database.
func GroupFiles ¶
GroupFiles combines pre-sorted .bygroup file data, by merging lines that share the same first 2 column values in to a single line.
func MergeAndCompress ¶
MergeAndCompress takes a list of open files, an open output file, and a Merger function to express the details of how the file contents should be merged. It compresses the output, and stores the merged input contents in there.
func MergeSortedFiles ¶
func MergeSortedFiles(inputs []string) (io.ReadCloser, func() error, error)
MergeSortedFiles shells out to `sort -m` to merge pre-sorted files together. Returns a pipe of the output from sort, and function you should call after you've finished reading the output to cleanup.
func MergeSummaryLines ¶
func MergeSummaryLines(data io.ReadCloser, matchColumns, summaryColumns int, mslm MatchingSummaryLineMerger, output io.Writer) error
MergeSummaryLines merges pre-sorted (pre-merged) summary data (eg. from a `sort -m` of .by* files), summing consecutive lines that have the same values in the first matchColumns columns, and outputting the results.
Types ¶
type MatchingSummaryLineMerger ¶
MatchingSummaryLineMerger is a func used by MergeSummaryLines() to handle summary columns when match columns match. a is the previous columns, b is the current. a should have its summary columns altered to merge information from b. Cols is the number of summary columns (the columns that contain info to eg. sum).
Source Files