obrc

package module

v0.0.0-...-6f1d7f0 Latest Latest Go to latest Published: Jan 3, 2024 License: MIT Imports: 6 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/clfs/obrc

Links

Open Source Insights

README ¶

obrc

An unoptimized, 2m9s One Billion Row Challenge solution in Go.

(I skipped the output sorting because I'm lazy.)

Usage

Generate a CSV:

go run ./cmd/gen -n 1000 > test.csv

Run the solver:

go run ./cmd/stats -input test.csv

Benchmark

MacBook Pro 14-inch, 2021, M1 Max, 64 GB:

$ go test -bench=. .
goos: darwin
goarch: arm64
pkg: github.com/clfs/obrc
BenchmarkGenerateCSV1e3-10       	    5095	    234506 ns/op
BenchmarkGenerateCSV1e4-10       	     512	   2333130 ns/op
BenchmarkGenerateCSV1e5-10       	      50	  23381521 ns/op
BenchmarkGenerateCSV1e6-10       	       5	 232976692 ns/op
BenchmarkTakeRecordings1e3-10    	    7417	    148938 ns/op
BenchmarkTakeRecordings1e4-10    	     903	   1321701 ns/op
BenchmarkTakeRecordings1e5-10    	      93	  12592483 ns/op
BenchmarkTakeRecordings1e6-10    	       9	 124397972 ns/op
PASS
ok  	github.com/clfs/obrc	13.300s

$ go run ./cmd/gen -n 1_000_000_000 > test.csv
$ time go run ./cmd/stats -input test.csv
...
... 126.41s user 3.52s system 100% cpu 2:09.61 total

Documentation ¶

Overview ¶

Package obrc implements the One Billion Row Challenge.

Index ¶

Variables
func Generate(w *csv.Writer, n int) error
func NewReader(r io.Reader) *csv.Reader
func NewWriter(w io.Writer) *csv.Writer
func TakeRecordings(r *csv.Reader) (map[string]Recording, error)
type Recording
- func NewRecording(measurement float64) Recording
type Station
- func RandStation() Station
- func (s *Station) Measure() float64

Constants ¶

This section is empty.

Variables ¶

View Source

var AllStations = []Station{}/* 413 elements not displayed */

AllStations contains all weather stations.

Functions ¶

func Generate ¶

func Generate(w *csv.Writer, n int) error

Generate writes n measurements to w.

func NewReader ¶

func NewReader(r io.Reader) *csv.Reader

NewReader returns a new csv.Reader with recommended settings.

func NewWriter ¶

func NewWriter(w io.Writer) *csv.Writer

NewWriter returns a new csv.Writer with recommended settings.

func TakeRecordings ¶

func TakeRecordings(r *csv.Reader) (map[string]Recording, error)

TakeRecordings takes a recording for each station.

Types ¶

type Recording ¶

type Recording struct {
	// contains filtered or unexported fields
}

Recording stores aggregate statistics about a weather station.

func NewRecording ¶

func NewRecording(measurement float64) Recording

NewRecording starts a new Recording.

func (*Recording) Add ¶

func (r *Recording) Add(measurement float64)

Add adds a measurement to the recording.

func (*Recording) Max ¶

func (r *Recording) Max() float64

Max returns the maximum measurement.

func (*Recording) Mean ¶

func (r *Recording) Mean() float64

Mean returns the mean measurement.

func (*Recording) Min ¶

func (r *Recording) Min() float64

Min returns the minimum measurement.

type Station ¶

type Station struct {
	Name     string
	MeanTemp float64
}

Station is a weather station.

func RandStation ¶

func RandStation() Station

RandStation returns a random weather station.

func (*Station) Measure ¶

func (s *Station) Measure() float64

Measure returns a random temperature for the station.

Source Files ¶

View all Source files

obrc.go

Directories ¶

Path	Synopsis
cmd
gen
stats

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL