The highest tagged major version is
README
¶
Text processing For example LF end-of-line, remove trailing white space, sort paragraphs. Go library and command. https://pkg.go.dev/github.com/MihaiB/textproc cd # go outside a module go get -u github.com/MihaiB/textproc/textproc `go env GOPATH`/bin/textproc -help
Documentation
¶
Overview ¶
Package textproc provides text processing.
Index ¶
- Variables
- func NewIoReader(r Reader) io.Reader
- func ReadAllTokens(r TokenReader) (tokens [][]rune, err error)
- func SendErrorAndClose(err error, ch chan<- error)
- func SendRunes(runes []rune, ch chan<- rune)
- func SendRunesAndClose(runes []rune, ch chan<- rune)
- func SendTokens(tokens [][]rune, ch chan<- []rune)
- func SendTokensAndClose(tokens [][]rune, ch chan<- []rune)
- type Reader
- func LFLines(r Reader) Reader
- func NewReader(r io.Reader) Reader
- func NewReaderFromRuneErrChan(runeCh <-chan rune, errCh <-chan error) Reader
- func NewReaderFromTokenReader(r TokenReader) Reader
- func NonEmptyFinalLF(r Reader) Reader
- func SortLFLinesI(r Reader) Reader
- func SortLFParagraphsI(r Reader) Reader
- func TrimLFTrailingSpace(r Reader) Reader
- func TrimLeadingLF(r Reader) Reader
- func TrimTrailingEmptyLFLines(r Reader) Reader
- type TokenReader
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidUTF8 = errors.New("Invalid UTF-8")
ErrInvalidUTF8 is the error returned when the input is not valid UTF-8.
Functions ¶
func NewIoReader ¶
NewIoReader returns a new io.Reader reading from r.
func ReadAllTokens ¶ added in v1.0.0
func ReadAllTokens(r TokenReader) (tokens [][]rune, err error)
ReadAllTokens reads tokens from r until it encounters an error and returns the tokens from all but the last call and the error from the last call.
func SendErrorAndClose ¶ added in v1.2.0
SendErrorAndClose sends the error on the channel then closes it.
func SendRunesAndClose ¶ added in v1.2.0
SendRunesAndClose sends the runes on the channel then closes it.
func SendTokens ¶ added in v1.2.0
SendTokens sends the tokens on the channel.
func SendTokensAndClose ¶ added in v1.2.0
SendTokensAndClose sends the tokens on the channel then closes it.
Types ¶
type Reader ¶
Reader reads runes.
func NewReader ¶
NewReader returns a new Reader reading from r. The new Reader returns ErrInvalidUTF8 if the input is not valid UTF-8.
func NewReaderFromRuneErrChan ¶ added in v1.2.0
NewReaderFromRuneErrChan returns a new Reader which reads all the runes then one error and panics if the error is nil.
func NewReaderFromTokenReader ¶ added in v1.0.0
func NewReaderFromTokenReader(r TokenReader) Reader
NewReaderFromTokenReader returns a new Reader reading from r.
func NonEmptyFinalLF ¶
NonEmptyFinalLF returns a new Reader which reads from r and ensures non-empty content ends with "\n".
func SortLFLinesI ¶ added in v1.1.0
SortLFLinesI returns a new Reader which reads the content of all lines from r using LFLineContent, sorts them in case-insensitive order and appends "\n" after each.
func SortLFParagraphsI ¶
SortLFParagraphsI returns a new Reader which reads the content of all paragraphs from r using LFParagraphContent, sorts them in case-insensitive order, joins them with "\n\n" and adds "\n" after the last paragraph.
func TrimLFTrailingSpace ¶
TrimLFTrailingSpace returns a new Reader which reads from r and removes white space at the end of lines. Lines are terminated by "\n".
func TrimLeadingLF ¶
TrimLeadingLF returns a new Reader which reads from r and removes "\n" characters at the start of the input.
func TrimTrailingEmptyLFLines ¶
TrimTrailingEmptyLFLines returns a new Reader which reads from r and removes empty lines at the end of the input. Lines are terminated by "\n".
type TokenReader ¶
TokenReader reads tokens. If err != nil callers must discard the returned token.
func LFLineContent ¶
func LFLineContent(r Reader) TokenReader
LFLineContent returns a new TokenReader reading the content of lines from r, excluding the line terminator "\n".
func LFParagraphContent ¶
func LFParagraphContent(r Reader) TokenReader
LFParagraphContent returns a new TokenReader reading the content of paragraphs from r, excluding the final line terminator. A paragraph consists of adjacent non-empty lines terminated by "\n".
func NewTokenReaderFromTokenErrChan ¶ added in v1.2.0
func NewTokenReaderFromTokenErrChan(tokenCh <-chan []rune, errCh <-chan error) TokenReader
NewTokenReaderFromTokenErrChan returns a new TokenReader which reads all the tokens then one error and panics if the error is nil.
func NewTokenReaderFromTokensErr ¶ added in v1.0.0
func NewTokenReaderFromTokensErr(tokens [][]rune, err error) TokenReader
NewTokenReaderFromTokensErr returns a new TokenReader which reads the tokens then returns err or panics if err is nil.
Source Files
Directories