scanner

package

v1.3.2 Latest Latest Go to latest Published: Jun 1, 2022 License: GPL-3.0 Imports: 5 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/0xrawsec/golang-utils

Links

Open Source Insights

Documentation ¶

Constants ¶

View Source

const (
	// EOF scanner
	EOF = -(iota + 1)
	// MaxTokenLen length of a token
	MaxTokenLen = 1 << 20 // 1 Mega Token
)

Variables ¶

View Source

var (
	// Whitespace default whitespaces characters taken by scanner
	Whitespace = "\t\n\r "
)

Functions ¶

This section is empty.

Types ¶

type Scanner ¶

type Scanner struct {
	Offset     int64
	LastOffset int64
	Whitespace *datastructs.BitSet
	Error      func(error)
	// contains filtered or unexported fields
}

Scanner structure definition

func New ¶

func New(r io.Reader) (s *Scanner)

New creates a new scanner from reader

func (*Scanner) InitWhitespace ¶

func (s *Scanner) InitWhitespace(w string)

InitWhitespace initialised an new set of whitespaces for the scanner

func (*Scanner) Scan ¶

func (s *Scanner) Scan() (r rune)

Scan scans until we reach a whitespace token

func (*Scanner) TokenText ¶

func (s *Scanner) TokenText() string

TokenText returns the string containing characters until the token was found

func (*Scanner) Tokenize ¶

func (s *Scanner) Tokenize() (cs chan string)

Tokenize returns a chan of tokens found in scanner until exhaustion

Source Files ¶

View all Source files

scanner.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL