streams

package module

v0.6.7 Latest Latest Go to latest Published: Jul 29, 2021 License: MIT Imports: 0 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/wangjuelong/go-streams

Links

Open Source Insights

README ¶

go-streams

A lightweight stream processing library for Go.
go-streams provides a simple and concise DSL to build data pipelines. pipeline-architecture-example

Wiki
In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements.

Overview

Building blocks:

Source - A Source is a set of stream processing steps that has one open output.
Flow - A Flow is a set of stream processing steps that has one open input and one open output.
Sink - A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.

Flow capabilities (flow package):

Map
FlatMap
Filter
PassThrough
Split
FanOut
Merge
Throttler
SlidingWindow
TumblingWindow

Supported Connectors:

Go channels
File system
Network (TCP, UDP)
WebSocket
Aerospike
Apache Kafka
Apache Pulsar
Redis

Examples

Usage samples are available in the examples directory.

License

Licensed under the MIT License.

Documentation ¶

Overview ¶

Package streams provides interfaces of the streaming library.

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Flow ¶

type Flow interface {
	Inlet
	Outlet
	Via(Flow) Flow
	To(Sink)
}

A Flow is a set of stream processing steps that has one open input and one open output.

type Inlet ¶

type Inlet interface {
	In() chan<- interface{}
}

Inlet is a type that exposes one open input. Implemented by the Flow and Sink.

type Outlet ¶

type Outlet interface {
	Out() <-chan interface{}
}

Outlet is a type that exposes one open output. Implemented by the Source and Flow.

type Sink ¶

type Sink interface {
	Inlet
}

A Sink is a set of stream processing steps that has one open input. Can be used as a Subscriber.

type Source ¶

type Source interface {
	Outlet
	Via(Flow) Flow
}

A Source is a set of stream processing steps that has one open output.

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
aerospike Package aerospike implements the Aerospike database connector.	Package aerospike implements the Aerospike database connector.
extension Package extension provides basic connector implementations.	Package extension provides basic connector implementations.
flow Package flow provides streams.Flow implementations.	Package flow provides streams.Flow implementations.
kafka Package kafka implements the Apache Kafka connector.	Package kafka implements the Apache Kafka connector.
nats Package nats implements the NATS Streaming connector.	Package nats implements the NATS Streaming connector.
pulsar Package pulsar implements the Apache Pulsar connector.	Package pulsar implements the Apache Pulsar connector.
redis Package redis implements the Redis Pub/Sub connector.	Package redis implements the Redis Pub/Sub connector.
util Package util provides utility functions.	Package util provides utility functions.
ospkg Package ospkg provides platform-specific utility functions and constants.	Package ospkg provides platform-specific utility functions and constants.
ws Package ws implements the WebSocket connector.	Package ws implements the WebSocket connector.

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL