velociraptor: Index | Files

package csv

import ""

Package csv reads and writes comma-separated values (CSV) files. There are many kinds of CSV files; this package supports the format described in RFC 4180.

A csv file contains zero or more records of one or more fields per record. Each record is separated by the newline character. The final record may optionally be followed by a newline character.


White space is considered part of a field.

Carriage returns before newline characters are silently removed.

Blank lines are ignored. A line with only whitespace characters (excluding the ending newline character) is not considered a blank line.

Fields which start and stop with the quote character " are called quoted-fields. The beginning and ending quote are not part of the field.

The source:

normal string,"quoted-field"

results in the fields

{`normal string`, `quoted-field`}

Within a quoted-field a quote character followed by a second quote character is considered a single quote.

"the ""word"" is true","a ""quoted-field"""

results in

{`the "word" is true`, `a "quoted-field"`}

Newlines and commas may be included in a quoted-field

field","comma is ,"

results in

	field`, `comma is ,`}

   Velociraptor - Hunting Evil
   Copyright (C) 2019 Velocidex Innovations.

   This program is free software: you can redistribute it and/or modify
   it under the terms of the GNU Affero General Public License as published
   by the Free Software Foundation, either version 3 of the License, or
   (at your option) any later version.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   GNU Affero General Public License for more details.

   You should have received a copy of the GNU Affero General Public License
   along with this program.  If not, see <>.


Package Files

doc.go reader.go utils.go writer.go


var (
    ErrTrailingComma = errors.New("extra delimiter at end of line") // Deprecated: No longer used.
    ErrBareQuote     = errors.New("bare \" in non-quoted-field")
    ErrQuote         = errors.New("extraneous or missing \" in quoted-field")
    ErrFieldCount    = errors.New("wrong number of fields")
    ErrNoLineFeed    = errors.New("line does not end with line feed")

These are the errors that can be returned in ParseError.Err.

func AnyToString Uses

func AnyToString(item vfilter.Any) string

type CSVReader Uses

type CSVReader chan *ordereddict.Dict

func GetCSVReader Uses

func GetCSVReader(fd file_store.FileReader) CSVReader

type CSVWriter Uses

type CSVWriter struct {
    // contains filtered or unexported fields

func GetCSVAppender Uses

func GetCSVAppender(scope *vfilter.Scope, fd io.Writer, write_headers bool) *CSVWriter

func GetCSVWriter Uses

func GetCSVWriter(scope *vfilter.Scope, fd file_store.FileWriter) (*CSVWriter, error)

func (*CSVWriter) Close Uses

func (self *CSVWriter) Close()

func (*CSVWriter) Write Uses

func (self *CSVWriter) Write(row vfilter.Row)

type ParseError Uses

type ParseError struct {
    StartLine int   // Line where the record starts
    Line      int   // Line where the error occurred
    Column    int   // Column (rune index) where the error occurred
    Err       error // The actual error

A ParseError is returned for parsing errors. Line numbers are 1-indexed and columns are 0-indexed.

func (*ParseError) Error Uses

func (e *ParseError) Error() string

type Reader Uses

type Reader struct {
    // Comma is the field delimiter.
    // It is set to comma (',') by NewReader.
    // Comma must be a valid rune and must not be \r, \n,
    // or the Unicode replacement character (0xFFFD).
    Comma rune

    // Comment, if not 0, is the comment character. Lines beginning with the
    // Comment character without preceding whitespace are ignored.
    // With leading whitespace the Comment character becomes part of the
    // field, even if TrimLeadingSpace is true.
    // Comment must be a valid rune and must not be \r, \n,
    // or the Unicode replacement character (0xFFFD).
    // It must also not be equal to Comma.
    Comment rune

    // FieldsPerRecord is the number of expected fields per record.
    // If FieldsPerRecord is positive, Read requires each record to
    // have the given number of fields. If FieldsPerRecord is 0, Read sets it to
    // the number of fields in the first record, so that future records must
    // have the same field count. If FieldsPerRecord is negative, no check is
    // made and records may have a variable number of fields.
    FieldsPerRecord int

    // If LazyQuotes is true, a quote may appear in an unquoted field and a
    // non-doubled quote may appear in a quoted field.
    LazyQuotes bool

    // If TrimLeadingSpace is true, leading white space in a field is ignored.
    // This is done even if the field delimiter, Comma, is white space.
    TrimLeadingSpace bool

    // ReuseRecord controls whether calls to Read may return a slice sharing
    // the backing array of the previous call's returned slice for performance.
    // By default, each call to Read returns newly allocated memory owned by the caller.
    ReuseRecord bool

    TrailingComma bool // Deprecated: No longer used.

    // If set we require that each line be terminated by a line
    // separator. This is important to be able to detect partially
    // written lines.
    RequireLineSeperator bool

    // Keep track of the number of bytes we read.
    ByteOffset int64
    // contains filtered or unexported fields

A Reader reads records from a CSV-encoded file.

As returned by NewReader, a Reader expects input conforming to RFC 4180. The exported fields can be changed to customize the details before the first call to Read or ReadAll.

The Reader converts all \r\n sequences in its input to plain \n, including in multiline field values, so that the returned data does not depend on which line-ending convention an input file uses.

func NewReader Uses

func NewReader(r io.ReadSeeker) *Reader

NewReader returns a new Reader that reads from r.

func (*Reader) Read Uses

func (r *Reader) Read() (record []string, err error)

Read reads one record (a slice of fields) from r. If the record has an unexpected number of fields, Read returns the record along with the error ErrFieldCount. Except for that case, Read always returns either a non-nil record or a non-nil error, but not both. If there is no data left to be read, Read returns nil, io.EOF. If ReuseRecord is true, the returned slice may be shared between multiple calls to Read.

func (*Reader) ReadAll Uses

func (r *Reader) ReadAll() (records [][]string, err error)

ReadAll reads all the remaining records from r. Each record is a slice of fields. A successful call returns err == nil, not err == io.EOF. Because ReadAll is defined to read until EOF, it does not treat end of file as an error to be reported.

func (*Reader) ReadAny Uses

func (r *Reader) ReadAny() ([]interface{}, error)

func (*Reader) Seek Uses

func (r *Reader) Seek(offset int64) error

type Writer Uses

type Writer struct {
    Comma   rune // Field delimiter (set to ',' by NewWriter)
    UseCRLF bool // True to use \r\n as the line terminator
    // contains filtered or unexported fields

A Writer writes records to a CSV encoded file.

As returned by NewWriter, a Writer writes records terminated by a newline and uses ',' as the field delimiter. The exported fields can be changed to customize the details before the first call to Write or WriteAll.

Comma is the field delimiter.

If UseCRLF is true, the Writer ends each output line with \r\n instead of \n.

func NewWriter Uses

func NewWriter(w io.Writer) *Writer

NewWriter returns a new Writer that writes to w.

func (*Writer) Error Uses

func (w *Writer) Error() error

Error reports any error that has occurred during a previous Write or Flush.

func (*Writer) Flush Uses

func (w *Writer) Flush()

Flush writes any buffered data to the underlying io.Writer. To check if an error occurred during the Flush, call Error.

func (*Writer) Write Uses

func (w *Writer) Write(record []string) error

Writer writes a single CSV record to w along with any necessary quoting. A record is a slice of strings with each string being one field.

func (*Writer) WriteAll Uses

func (w *Writer) WriteAll(records [][]string) error

WriteAll writes multiple CSV records to w using Write and then calls Flush.

func (*Writer) WriteAny Uses

func (w *Writer) WriteAny(record []interface{}) error

Package csv imports 17 packages (graph) and is imported by 18 packages. Updated 2020-04-06. Refresh now. Tools for package owners.