stream

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 19, 2020 License: MIT Imports: 11 Imported by: 1

README

go-stream

Open In GitPod GitHub code size in bytes GitHub commit activity GitHub last commit

utility library with different implementations of I/O primitives for various common tasks.This package is experimental. use at your own risk.

  • HashWriter an io.Writer that stores hash of data as it is getting passed through it. It can use multiple hash.Hash types to calculate hash of the stream.

Benchmark

  • HashWriter benchmarks with standalone stdlib md5 and sha256 hash functions , HashWriter backed by stdlib md5 and sha256 hash functions and minio's implementation of md5 and sha256 hash functions (go version go1.15 linux/amd64)
Running tool: /home/gitpod/go/bin/go test -benchmem -run=^$ github.com/da-moon/go-stream -bench ^(BenchmarkHashWriter)$ -v
goos: linux
goarch: amd64
pkg: github.com/da-moon/go-stream
BenchmarkHashWriter
BenchmarkHashWriter/8
BenchmarkHashWriter/8/single-hash
BenchmarkHashWriter/8/single-hash/SHA256
BenchmarkHashWriter/8/single-hash/SHA256/stdlib
BenchmarkHashWriter/8/single-hash/SHA256/stdlib-16         	 4466779	       265 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/8/single-hash/SHA256/hashwriter-stdlib
BenchmarkHashWriter/8/single-hash/SHA256/hashwriter-stdlib-16         	 2564110	       453 ns/op	      48 B/op	       1 allocs/op
BenchmarkHashWriter/8/single-hash/SHA256/hashwriter-minio
BenchmarkHashWriter/8/single-hash/SHA256/hashwriter-minio-16          	 2858588	       433 ns/op	      61 B/op	       1 allocs/op
BenchmarkHashWriter/8/single-hash/MD5
BenchmarkHashWriter/8/single-hash/MD5/stdlib
BenchmarkHashWriter/8/single-hash/MD5/stdlib-16                       	 8055064	       151 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/8/single-hash/MD5/hashwriter-stdlib
BenchmarkHashWriter/8/single-hash/MD5/hashwriter-stdlib-16            	  483354	      2642 ns/op	     546 B/op	      13 allocs/op
BenchmarkHashWriter/8/single-hash/MD5/hashwriter-minio
BenchmarkHashWriter/8/single-hash/MD5/hashwriter-minio-16             	  309547	      4026 ns/op	     545 B/op	      13 allocs/op
BenchmarkHashWriter/8/multi-hash
BenchmarkHashWriter/8/multi-hash/stdlib
BenchmarkHashWriter/8/multi-hash/stdlib-16                            	 2363575	       525 ns/op	      48 B/op	       2 allocs/op
BenchmarkHashWriter/8/multi-hash/hashwriter
BenchmarkHashWriter/8/multi-hash/hashwriter-16                        	 1000000	      1022 ns/op	      50 B/op	       1 allocs/op
BenchmarkHashWriter/1024
BenchmarkHashWriter/1024/single-hash
BenchmarkHashWriter/1024/single-hash/SHA256
BenchmarkHashWriter/1024/single-hash/SHA256/stdlib
BenchmarkHashWriter/1024/single-hash/SHA256/stdlib-16                 	  363068	      3343 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/1024/single-hash/SHA256/hashwriter-stdlib
BenchmarkHashWriter/1024/single-hash/SHA256/hashwriter-stdlib-16      	  270160	      7156 ns/op	    2081 B/op	       1 allocs/op
BenchmarkHashWriter/1024/single-hash/SHA256/hashwriter-minio
BenchmarkHashWriter/1024/single-hash/SHA256/hashwriter-minio-16       	  276546	     29231 ns/op	    4036 B/op	       1 allocs/op
BenchmarkHashWriter/1024/single-hash/MD5
BenchmarkHashWriter/1024/single-hash/MD5/stdlib
BenchmarkHashWriter/1024/single-hash/MD5/stdlib-16                    	  670987	      1921 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/1024/single-hash/MD5/hashwriter-stdlib
BenchmarkHashWriter/1024/single-hash/MD5/hashwriter-stdlib-16         	  222818	      5038 ns/op	    2967 B/op	      13 allocs/op
BenchmarkHashWriter/1024/single-hash/MD5/hashwriter-minio
BenchmarkHashWriter/1024/single-hash/MD5/hashwriter-minio-16          	  119022	      9511 ns/op	    3688 B/op	      20 allocs/op
BenchmarkHashWriter/1024/multi-hash
BenchmarkHashWriter/1024/multi-hash/stdlib
BenchmarkHashWriter/1024/multi-hash/stdlib-16                         	  209500	      5610 ns/op	      48 B/op	       2 allocs/op
BenchmarkHashWriter/1024/multi-hash/hashwriter
BenchmarkHashWriter/1024/multi-hash/hashwriter-16                     	  120184	      9780 ns/op	    3160 B/op	       8 allocs/op
BenchmarkHashWriter/8192
BenchmarkHashWriter/8192/single-hash
BenchmarkHashWriter/8192/single-hash/SHA256
BenchmarkHashWriter/8192/single-hash/SHA256/stdlib
BenchmarkHashWriter/8192/single-hash/SHA256/stdlib-16                 	   48978	     25139 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/8192/single-hash/SHA256/hashwriter-stdlib
BenchmarkHashWriter/8192/single-hash/SHA256/hashwriter-stdlib-16      	   32734	     31755 ns/op	   16494 B/op	       1 allocs/op
BenchmarkHashWriter/8192/single-hash/SHA256/hashwriter-minio
BenchmarkHashWriter/8192/single-hash/SHA256/hashwriter-minio-16       	   38716	     44113 ns/op	   27871 B/op	       1 allocs/op
BenchmarkHashWriter/8192/single-hash/MD5
BenchmarkHashWriter/8192/single-hash/MD5/stdlib
BenchmarkHashWriter/8192/single-hash/MD5/stdlib-16                    	   82287	     14096 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashWriter/8192/single-hash/MD5/hashwriter-stdlib
BenchmarkHashWriter/8192/single-hash/MD5/hashwriter-stdlib-16         	   47614	     24452 ns/op	   23113 B/op	      13 allocs/op
BenchmarkHashWriter/8192/single-hash/MD5/hashwriter-minio
BenchmarkHashWriter/8192/single-hash/MD5/hashwriter-minio-16          	   35079	     36130 ns/op	   32095 B/op	      20 allocs/op
BenchmarkHashWriter/8192/multi-hash
BenchmarkHashWriter/8192/multi-hash/stdlib
BenchmarkHashWriter/8192/multi-hash/stdlib-16                         	   29565	     39082 ns/op	      48 B/op	       2 allocs/op
BenchmarkHashWriter/8192/multi-hash/hashwriter
BenchmarkHashWriter/8192/multi-hash/hashwriter-16                     	   22287	     49527 ns/op	   25075 B/op	       8 allocs/op
PASS
ok  	github.com/da-moon/go-stream	41.723s

Documentation

Overview

Package stream implements different io interfaces to help with common operations

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ShutdownMD5Hasher 

func ShutdownMD5Hasher()

ShutdownMD5Hasher must be called when we are done with using the md5hasher

Types

type HashAlgorithm 

type HashAlgorithm string

HashAlgorithm provides a convenient way to have consistency when accessing default hash algorithms 

var (
	// SHA256 ...
	SHA256 HashAlgorithm = "SHA256"
	// MD5 ...
	MD5 HashAlgorithm = "MD5"
)

list of hash algorithms with helper methods

func NewHashAlgorithm 

func NewHashAlgorithm(arg string) HashAlgorithm

NewHashAlgorithm is a utility function used for casting a string to hash algorithm enum

type HashWriter 

type HashWriter interface {
	io.Writer
	Hash(h HashAlgorithm) ([]byte, error)
	HexString(h HashAlgorithm) (string, error)
	Base64String(h HashAlgorithm) (string, error)
	Reset()
}

HashWriter implements an io.writer that hashes the payload as operating on data while any hash.Hasher can be used as underlying hashing algorithm, at this point utility functions for md5 and sha256n are written

func NewHashWriter 

func NewHashWriter(writer io.Writer, opts ...HashWriterOption) (HashWriter, error)

NewHashWriter ...

type HashWriterOption 

type HashWriterOption func(*hashWriter) error

HashWriterOption ...

func WithHasher 

func WithHasher(algorithm HashAlgorithm, hasher hash.Hash) HashWriterOption

WithHasher sets underlying hash algorithms [WARN] => make sure s.hasher is allocated in memory before calling this function

func WithLogger 

func WithLogger(arg *log.Logger) HashWriterOption

WithLogger ...

func WithMD5 

func WithMD5() HashWriterOption

WithMD5 is a sets up md5 hashing with minio's md5 Hasher.

func WithSHA256 

func WithSHA256() HashWriterOption

WithSHA256 is a convenience method that adds sha256 hashing based on minio's library.

func WithWrappedLogger 

func WithWrappedLogger(arg *logger.WrappedLogger) HashWriterOption

WithWrappedLogger ...

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.