dedup

package
v0.26.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 8, 2024 License: MIT Imports: 3 Imported by: 2

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func KeyFromCSV 

func KeyFromCSV(b []byte, fields []int, sep string) string

KeyFromCSV generates a 'key' of fields values by concatenating the field values in the order received in fields.

If returned string is empty then the fields were not found.

If a fields index value is out of range then that field is ignored and not included in the key.

func KeyFromJSON 

func KeyFromJSON(b []byte, fields []string) string

KeyFromJSON generates a 'key' of fields values by concatenating the field values in the order received in fields.

If returned string is empty then the fields were not found.

Types

type Dedup 

type Dedup struct {
	// contains filtered or unexported fields
}

Dedup will dedup lines added to AddLine by the provided key. Newer lines replace older lines. If order is important then consider sorting all the lines first.

func New 

func New() *Dedup

func (*Dedup) Add 

func (w *Dedup) Add(key string, b []byte)

AddLine will add the bytes record b to the pool of deduped linesMp. As a help, basic csv and json key generators are provided as standalone functions in this package.

func (*Dedup) Lines 

func (w *Dedup) Lines() [][]byte

Lines returns the deduped lines.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.