README
¶
backscanner
Ever needed or wondered how to efficiently search for something in a log file, but starting at the end and going backward? Here's your solution.
Package backscanner provides a scanner similar to bufio.Scanner, but it reads
and returns lines in reverse order, starting at a given position (which may be
the end of the input) and going backward.
This library only uses the standard library, but the test uses an external package. To install this library (along with the test dependency), simply run:
go get -t github.com/icza/backscanner
Advancing and accessing lines of the input is done by calling Scanner.Line(),
which returns the next line (previous in the source) as a string.
For maximum efficiency there is Scanner.LineBytes(). It returns the next line
as a byte slice, which shares its backing array with the internal buffer of
Scanner. This is because no copy is made from the line data; but this also
means you can only inspect or search in the slice before calling Line() or
LineBytes() again, as the content of the internal buffer–and thus slices
returned by LineBytes()–may be overwritten. If you need to retain the line
data, make a copy of it or use Line().
Example using it:
input := "Line1\nLine2\nLine3"
scanner := backscanner.New(strings.NewReader(input), len(input))
for {
line, pos, err := scanner.Line()
if err != nil {
fmt.Println("Error:", err)
break
}
fmt.Printf("Line position: %2d, line: %q\n", pos, line)
}
Output:
Line position: 12, line: "Line3"
Line position: 6, line: "Line2"
Line position: 0, line: "Line1"
Error: EOF
Using it to efficiently scan a file, finding last occurrence of a string ("error"):
file, err := os.Open("mylog.txt")
if err != nil {
panic(err)
}
fileStatus, err := file.Stat()
if err != nil {
panic(err)
}
defer file.Close()
scanner := backscanner.New(file, int(fileStatus.Size()))
what := []byte("error")
for {
line, pos, err := scanner.LineBytes()
if err != nil {
if err == io.EOF {
fmt.Printf("%q is not found in file.\n", what)
} else {
fmt.Println("Error:", err)
}
break
}
if bytes.Contains(line, what) {
fmt.Printf("Found %q at line position: %d, line: %q\n", what, pos, line)
break
}
}
Documentation
¶
Overview ¶
Package backscanner provides a scanner similar to bufio.Scanner, but it reads and returns lines in reverse order, starting at a given position (which may be the end of the input) and going backward.
Unlike with bufio.Scanner, max line length may be configured.
Advancing and accessing lines of the input is done by calling Scanner.Line(), which returns the next line (previous in the source) as a string.
For maximum efficiency there is Scanner.LineBytes(). It returns the next line as a byte slice, which shares its backing array with the internal buffer of Scanner. This is because no copy is made from the line data; but this also means you can only inspect or search in the slice before calling Line() or LineBytes() again, as the content of the internal buffer–and thus slices returned by LineBytes()–may be overwritten. If you need to retain the line data, make a copy of it or use Line().
Example using it:
input := "Line1\nLine2\nLine3"
scanner := backscanner.New(strings.NewReader(input), len(input))
for {
line, pos, err := scanner.Line()
if err != nil {
fmt.Println("Error:", err)
break
}
fmt.Printf("Line position: %2d, line: %q\n", pos, line)
}
Output:
Line position: 12, line: "Line3" Line position: 6, line: "Line2" Line position: 0, line: "Line1" Error: EOF
Using it to efficiently scan a file, finding last occurrence of a string ("error"):
f, err := os.Open("mylog.txt")
if err != nil {
panic(err)
}
fi, err := f.Stat()
if err != nil {
panic(err)
}
defer f.Close()
scanner := backscanner.New(f, int(fi.Size()))
what := []byte("error")
for {
line, pos, err := scanner.LineBytes()
if err != nil {
if err == io.EOF {
fmt.Printf("%q is not found in file.\n", what)
} else {
fmt.Println("Error:", err)
}
break
}
if bytes.Contains(line, what) {
fmt.Printf("Found %q at line position: %d, line: %q\n", what, pos, line)
break
}
}
Index ¶
Constants ¶
const ( // DefaultChunkSize is the default value for the ChunkSize option DefaultChunkSize = 1024 // DefaultMaxBufferSize is the default value for the MaxBufferSize option DefaultMaxBufferSize = 1 << 20 // 1 MB )
Variables ¶
var ( // ErrLongLine indicates that the line is longer than the internal buffer size ErrLongLine = errors.New("line too long") )
Functions ¶
This section is empty.
Types ¶
type Options ¶
type Options struct {
// ChunkSize specifies the size of the chunk that is read at once from the input.
ChunkSize int
// MaxBufferSize limits the maximum size of the buffer used internally.
// This also limits the max line size.
MaxBufferSize int
}
Options contains parameters that influence the internal working of the Scanner.
type Scanner ¶
type Scanner struct {
// contains filtered or unexported fields
}
Scanner is the back-scanner implementation.
func NewOptions ¶
NewOptions returns a new Scanner with the given Options. Invalid option values are replaced with their default values.
func (*Scanner) Line ¶
Line returns the next line from the input and its absolute byte-position. Line ending is cut from the line. Empty lines are also returned. After returning the last line (which is the first in the input), subsequent calls report io.EOF.
func (*Scanner) LineBytes ¶
LineBytes returns the bytes of the next line from the input and its absolute byte-position. Line ending is cut from the line. Empty lines are also returned. After returning the last line (which is the first in the input), subsequent calls report io.EOF.
This method is for efficiency if you need to inspect or search in the line. The returned line slice shares data with the internal buffer of the Scanner, and its content may be overwritten in subsequent calls to LineBytes() or Line(). If you need to retain the line data, make a copy of it or use the Line() method.
Source Files