README
¶
batcher is a package for batching individual requests into batch
requests.
For some live services, batching is necessary to maximize throughput. In particular, services that require disk I/O or leverages GPUs would require batching as the overhead cost associated to each request is long and near constant. The typical way that batching is achieved is with the help of a queue system.
batcher takes this design pattern and formalizes it into a template for
developers to conveniently incorporate batching to their live services.
Installation and Docs
Within of a module, this package can be installed with go get.
go get github.com/mingruimingrui/batcher
Auto-generated documentation is available at https://pkg.go.dev/github.com/mingruimingrui/batcher.
Usage
Usage of this library typically begins with creation of a new RequestBatcher
variable.
Typically this is done on the global scope.
var (
requestBatcher *batcher.RequestBatcher
...
)
...
func main() {
requestBatcher = batcher.RequestBatcher(
context.Background(),
&batcher.BatchingConfig{
MaxBatchSize: ...,
BatchTimeout: ...,
// SendF is a function that handles a batch of requests
SendF: func(body *[]interface{}) (*[]interface{}, error) {
...
}
},
)
...
}
To submit a request, simply use the SendRequestWithTimeout method
resp, err := requestBatcher.SendRequestWithTimeout(&someRequestBody, someTimeoutDuration)
When multiple requests are send together from multiple goroutines,
the RequestBatcher would group those requests together into a single batch
so SendF would be called minimally.
BatchingConfig
batcher.BatchingConfig controls batching behavior and accepts the
following parameters.
-
MaxBatchSize{int}
The maximum number of requests per batch -
BatchTimeout{time.Duration}
The maximum wait time before batch is sent -
SendF{func(body *[]interface{}) (*[]interface{}, error)}
A function for the user to define how to handle a batch
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // Version -- Current version of this package Version string = "1.0.1" )
Functions ¶
This section is empty.
Types ¶
type BatchingConfig ¶
type BatchingConfig struct {
// Maximum request size of each batch.
MaxBatchSize int
// Maximum wait time before batch should be executed.
BatchTimeout time.Duration
// User defined SendF for sending a batch request.
// See SendFunc for type definition of this function.
SendF SendFunc
}
BatchingConfig determines how batching is done in a RequestBatcher.
type RequestBatcher ¶
type RequestBatcher struct {
sync.Mutex
*BatchingConfig
// contains filtered or unexported fields
}
RequestBatcher handles receiving of new requests, and all the background asynchronous tasks to batch and send batch.
A new RequestBatcher should be created using the NewRequestBatcher function.
Expected usage pattern of the RequestBatcher involves declaring a RequestBatcher on global scope and calling SendRequestWithTimeout from multiple goroutines.
func NewRequestBatcher ¶
func NewRequestBatcher( ctx context.Context, config *BatchingConfig, ) *RequestBatcher
NewRequestBatcher creates a new RequestBatcher from a Context and a BatchingConfig.
In the typical usage pattern, a RequestBatcher should always be alive so it is safe and recommended to use the background context.
func (*RequestBatcher) SendRequestWithTimeout ¶
func (b *RequestBatcher) SendRequestWithTimeout( newRequestBody *interface{}, timeout time.Duration, ) (interface{}, error)
SendRequestWithTimeout is a method to make a single request. It manages registering the request into the batcher, and waiting on the response.
Arguments:
newRequestBody {*interface{}} -- A request body. SendF will expect
a slice of objects like newRequestBody.
Returns:
interface{} -- A response body. SendF's output is expected to be a slice
of objects like this.
error -- Error
Source Files
Directories