gopool

package module
v1.2.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 23, 2020 License: MIT Imports: 4 Imported by: 4

README

gopool

Gopool offers a way to maintain a free-list, or a pool of resources in Go programs.

Description

It is often the case that resource setup and teardown can be quite demanding, and it is desirable to reuse resources rather than close them and create new ones when needed. Two such examples are network sockets to a given peer, and large byte buffers for building query responses.

Although most customizations are optional, it does require specification of a customized setup function to create new resources. Optional resources include specifying the size of the resource pool, specifying a per-use reset function, and specifying a close function to be called when the pool is no longer needed. The close function is called one time for each resource in the pool, with that resource as the close function's sole argument.

Usage

Documentation is available via GoDoc.

Example

The most basic example is creating a buffer pool and using it.

WARNING: You Must ensure resource returns to pool otherwise gopool will deadlock once all resources used. If you use the resource in a function, consider using defer pool.Put(bb) immediately after you obtain the resource at the top of your function.

    package main

    import (
        "bytes"
        "errors"
        "fmt"
        "log"
        "math/rand"
        "sync"

        "github.com/karrick/gopool"
    )

    const (
        bufSize  = 64 * 1024
        poolSize = 25
    )

    func main() {
        const iterationCount = 1000
        const parallelCount = 100

        makeBuffer := func() (interface{}, error) {
            return bytes.NewBuffer(make([]byte, 0, bufSize)), nil
        }

        resetBuffer := func(item interface{}) {
            item.(*bytes.Buffer).Reset()
        }

        bp, err := gopool.New(gopool.Size(poolSize), gopool.Factory(makeBuffer), gopool.Reset(resetBuffer))
        if err != nil {
            log.Fatal(err)
        }

        var wg sync.WaitGroup
        wg.Add(parallelCount)

        for i := 0; i < parallelCount; i++ {
            go func() {
                defer wg.Done()

                for j := 0; j < iterationCount; j++ {
                    if err := grabBufferAndUseIt(bp); err != nil {
                        fmt.Println(err)
                        return
                    }
                }
            }()
        }
        wg.Wait()
    }

    func grabBufferAndUseIt(pool gopool.Pool) error {
        // WARNING: Must ensure resource returns to pool otherwise gopool will deadlock once all
        // resources used.
        bb := pool.Get().(*bytes.Buffer)
        defer pool.Put(bb) // IMPORTANT: defer here to ensure invoked even when subsequent code bails

        for k := 0; k < bufSize; k++ {
            if rand.Intn(100000000) == 1 {
                return errors.New("random error to illustrate need to return resource to pool")
            }
            bb.WriteByte(byte(k % 256))
        }
        return nil
    }

Documentation

Index

Constants

View Source
const DefaultSize = 10

DefaultSize is the default number of items that will be maintained in the pool.

Variables

This section is empty.

Functions

This section is empty.

Types

type ArrayPool added in v1.1.0

type ArrayPool struct {
	// contains filtered or unexported fields
}

ArrayPool implements the Pool interface, maintaining a pool of resources.

func (*ArrayPool) Close added in v1.1.0

func (pool *ArrayPool) Close() error

Close is called when the Pool is no longer needed, and the resources in the Pool ought to be released. If a Pool has a close function, it will be invoked one time for each resource, with that resource as its sole argument.

func (*ArrayPool) Get added in v1.1.0

func (pool *ArrayPool) Get() interface{}

func (*ArrayPool) Put added in v1.1.0

func (pool *ArrayPool) Put(item interface{})

Put will release a resource back to the pool. Put blocks if pool already full. If the Pool was initialized with a Reset function, it will be invoked with the resource as its sole argument, prior to the resource being added back to the pool. If Put is called when adding the resource to the pool _would_ result in having more elements in the pool than the pool size, the resource is effectively dropped on the floor after calling any optional Reset and Close methods on the resource.

type ChanPool

type ChanPool struct {
	// contains filtered or unexported fields
}

ChanPool implements the Pool interface, maintaining a pool of resources.

func (*ChanPool) Close

func (pool *ChanPool) Close() error

Close is called when the Pool is no longer needed, and the resources in the Pool ought to be released. If a Pool has a close function, it will be invoked one time for each resource, with that resource as its sole argument.

func (*ChanPool) Get

func (pool *ChanPool) Get() interface{}

Get acquires and returns an item from the pool of resources. Get blocks while there are no items in the pool.

func (*ChanPool) Put

func (pool *ChanPool) Put(item interface{})

Put will release a resource back to the pool. Put blocks if pool already full. If the Pool was initialized with a Reset function, it will be invoked with the resource as its sole argument, prior to the resource being added back to the pool. If Put is called when adding the resource to the pool _would_ result in having more elements in the pool than the pool size, the resource is effectively dropped on the floor after calling any optional Reset and Close methods on the resource.

type Configurator

type Configurator func(*config) error

Configurator is a function that modifies a pool configuration structure.

func Close

func Close(close func(interface{}) error) Configurator

Close specifies the optional function to be called once for each resource when the Pool is closed.

func Factory

func Factory(factory func() (interface{}, error)) Configurator

Factory specifies the function used to make new elements for the pool. The factory function is called to fill the pool N times during initialization, for a pool size of N.

func Reset

func Reset(reset func(interface{})) Configurator

Reset specifies the optional function to be called on resources when released back to the pool. If a reset function is not specified, then resources are returned to the pool without any reset step. For instance, if maintaining a Pool of buffers, a library may choose to have the reset function invoke the buffer's Reset method to free resources prior to returning the buffer to the Pool.

func Size

func Size(size int) Configurator

Size specifies the number of items to maintain in the pool.

type Pool

type Pool interface {
	Close() error
	Get() interface{}
	Put(interface{})
}

Pool is the interface implemented by an object that acts as a free-list resource pool.

func New

func New(setters ...Configurator) (Pool, error)

New creates a new Pool. The factory method used to create new items for the Pool must be specified using the gopool.Factory method. Optionally, the pool size and a reset function can be specified.

package main

import (
	"bytes"
	"errors"
	"fmt"
	"log"
	"math/rand"
	"sync"

	"github.com/karrick/gopool"
)

const (
	bufSize  = 64 * 1024
	poolSize = 25
)

func main() {
	const iterationCount = 1000
	const parallelCount = 100

	makeBuffer := func() (interface{}, error) {
		return bytes.NewBuffer(make([]byte, 0, bufSize)), nil
	}

	resetBuffer := func(item interface{}) {
		item.(*bytes.Buffer).Reset()
	}

	bp, err := gopool.New(gopool.Size(poolSize), gopool.Factory(makeBuffer), gopool.Reset(resetBuffer))
	if err != nil {
		log.Fatal(err)
	}

	var wg sync.WaitGroup
	wg.Add(parallelCount)

	for i := 0; i < parallelCount; i++ {
		go func() {
			defer wg.Done()

			for j := 0; j < iterationCount; j++ {
				if err := grabBufferAndUseIt(bp); err != nil {
					fmt.Println(err)
					return
				}
			}
		}()
	}
	wg.Wait()
}

func grabBufferAndUseIt(pool gopool.Pool) error {
	// WARNING: Must ensure resource returns to pool otherwise gopool will deadlock once all
	// resources used.
	bb := pool.Get().(*bytes.Buffer)
	defer pool.Put(bb) // IMPORTANT: defer here to ensure invoked even when subsequent code bails

	for k := 0; k < bufSize; k++ {
		if rand.Intn(100000000) == 1 {
			return errors.New("random error to illustrate need to return resource to pool")
		}
		bb.WriteByte(byte(k % 256))
	}
	return nil
}

func NewArrayPool added in v1.1.0

func NewArrayPool(setters ...Configurator) (Pool, error)

NewArrayPool creates a new Pool. The factory method used to create new items for the Pool must be specified using the gopool.Factory method. Optionally, the pool size and a reset function can be specified.

package main

import (
	"bytes"
	"errors"
	"fmt"
	"log"
	"math/rand"
	"sync"

	"github.com/karrick/gopool"
)

const (
	bufSize  = 64 * 1024
	poolSize = 25
)

func main() {
	const iterationCount = 1000
	const parallelCount = 100

	makeBuffer := func() (interface{}, error) {
		return bytes.NewBuffer(make([]byte, 0, bufSize)), nil
	}

	resetBuffer := func(item interface{}) {
		item.(*bytes.Buffer).Reset()
	}

	bp, err := gopool.NewArrayPool(gopool.Size(poolSize), gopool.Factory(makeBuffer), gopool.Reset(resetBuffer))
	if err != nil {
		log.Fatal(err)
	}

	var wg sync.WaitGroup
	wg.Add(parallelCount)

	for i := 0; i < parallelCount; i++ {
		go func() {
			defer wg.Done()

			for j := 0; j < iterationCount; j++ {
				if err := grabBufferAndUseIt(bp); err != nil {
					fmt.Println(err)
					return
				}
			}
		}()
	}
	wg.Wait()
}

func grabBufferAndUseIt(pool gopool.Pool) error {
	// WARNING: Must ensure resource returns to pool otherwise gopool will deadlock once all
	// resources used.
	bb := pool.Get().(*bytes.Buffer)
	defer pool.Put(bb) // IMPORTANT: defer here to ensure invoked even when subsequent code bails

	for k := 0; k < bufSize; k++ {
		if rand.Intn(100000000) == 1 {
			return errors.New("random error to illustrate need to return resource to pool")
		}
		bb.WriteByte(byte(k % 256))
	}
	return nil
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL