README
¶
CGOAlloc
Reducing Malloc/Free traffic to cgo
Why?
Cgo overhead is a little higher than many are comfortable with (at the time of this writing, a simple call tends to run between 4-6x an equivalent JNI call). Where they really get you, though, is the data marshalling. Each individual call to malloc or free is another cgo call with a 30-50ns overhead.
This library provides an Allocator interface which can be used to provide alternative allocators to C.malloc and C.free. It also provides a Destroy method, which will clean up any overhead allocated via cgo, as well as make a best-effort to panic if any memory has been allocated and not freed via the destroyed Allocator. This functionality uses whatever information the allocator in question happens to have available, so it should not be considered definitive.
More importantly, it provides an allocator FixedBlockAllocator which sits on top of another Allocator and allows you to malloc large buffers that are doled out in blocks, amortizing the malloc and free calls across the life of a program.
Also available:
DefaultAllocator- calls cgo for Malloc and FreeFallbackAllocator- Accepts a FixedBlockAllocator and one other allocator- if the malloc can fit in the FBA, it uses that, otherwise it mallocs in the other allocator. You can use this to fall back on the default allocator for large requests. You could also use several to set up a multi-tiered FBA, I suppose.ArenaAllocator- sits on top of another allocator. Exposes a FreeAll method which will free all memory allocated through the ArenaAllocator. ArenaAllocator is optimized forFreeAlland ordinary frees have a cost of O(N)
Are these thread-safe?
The DefaultAllocator is! And as slow as cgo is, it's still far faster than any locking mechanism in existence, so if you need thread safety, that's what you should use.
What's the performance like?
In terms of memory overhead, it's kind of bad! I use a lot of maps and slices to track allocated-but-not-freed data. In terms of speed:
Default cgo
BenchmarkDefaultTemporaryData
BenchmarkDefaultTemporaryData-16 12792590 94.58 ns/op
BenchmarkDefaultGrowShrink
BenchmarkDefaultGrowShrink-16 11286946 104.7 ns/op
Fixed Buffer
BenchmarkFBATemporaryData
BenchmarkFBATemporaryData-16 123561244 9.714 ns/op
BenchmarkFBAGrowShrink
BenchmarkFBAGrowShrink-16 64682006 34.83 ns/op
3-Layer Fallback
BenchmarkMultilayerTemporaryData
BenchmarkMultilayerTemporaryData-16 72288720 17.06 ns/op
BenchmarkMultilayerGrowShrink
BenchmarkMultilayerGrowShrink-16 48367983 35.78 ns/op
Arena
BenchmarkArenaTemporaryData
BenchmarkArenaTemporaryData-16 40963460 29.24 ns/op
"It's fine!"
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Allocator ¶
type Allocator interface {
// Malloc is equivalent to C.Malloc
Malloc(size int) unsafe.Pointer
// Free is equivalent to C.Free
Free(pointer unsafe.Pointer)
// Destroy frees any backing resources that require it and throws an error if it detects that any memory has not been
// freed. This leak check is best-effort- no additional instrumentation exists to ensure it is correct, so it may be
// more or less accurate with different Allocator implementations
Destroy() error
}
Allocator is the base interface of cgoalloc- libraries that want to make use of cgoalloc should arrange for their methods to accept an Allocator at runtime and use the interface's Malloc/Free to interact with memory. (cgoalloc.CString and cgoalloc.CBytes provide similar functionality to their C.* equivalents, but accept an Allocator).
Executable packages that want to make use of cgoalloc should initialize one or more implementation of Allocator and use them- when finished with an Allocator it's useful to call Destroy. This will ensure that any C memory pages will be deallocated, and it will also attempt to return an error if any Malloc has not been paired with a Free
type ArenaAllocator ¶
type ArenaAllocator struct {
// contains filtered or unexported fields
}
ArenaAllocator is an Allocator implementation which accepts an Allocator object and passes Malloc and Free calls to the underlying Allocator. However, it exposes a FreeAll method which will instantly free any Malloc calls which have been proxied through the ArenaAllocator. Because FreeAll requires all Malloc calls to be tracked in the ArenaAllocator, and because that tracking is optimized for FreeAll speed, direct calls to Free have O(N) time and are not recommended.
ArenaAllocator is intended to be spun up temporarily for a flurry of malloc activity that then needs to be undone at the end.
func CreateArenaAllocator ¶
func CreateArenaAllocator(inner Allocator) *ArenaAllocator
func (*ArenaAllocator) Destroy ¶
func (a *ArenaAllocator) Destroy() error
func (*ArenaAllocator) Free ¶
func (a *ArenaAllocator) Free(ptr unsafe.Pointer)
func (*ArenaAllocator) FreeAll ¶
func (a *ArenaAllocator) FreeAll()
FreeAll calls Free for every pointer allocated but not freed through this ArenaAllocator
type DefaultAllocator ¶
type DefaultAllocator struct{}
DefaultAllocator is an Allocator implementation that just calls C.malloc/C.free
func (*DefaultAllocator) Destroy ¶
func (a *DefaultAllocator) Destroy() error
func (*DefaultAllocator) Free ¶
func (a *DefaultAllocator) Free(pointer unsafe.Pointer)
type FallbackAllocator ¶ added in v1.1.0
type FallbackAllocator struct {
// contains filtered or unexported fields
}
FallbackAllocator is an Allocator implementation which accepts a FixedBlockAllocator and sends all Malloc calls which can fit in the FBA's block size to that FixedBlockAllocator. All other calls are sent to a fallback allocator.
func CreateFallbackAllocator ¶ added in v1.1.0
func CreateFallbackAllocator(fixedBlock FixedBlockAllocator, fallback Allocator) *FallbackAllocator
func (*FallbackAllocator) Destroy ¶ added in v1.1.0
func (a *FallbackAllocator) Destroy() error
func (*FallbackAllocator) Free ¶ added in v1.1.0
func (a *FallbackAllocator) Free(ptr unsafe.Pointer)
type FixedBlockAllocator ¶
type FixedBlockAllocator interface {
Allocator
// contains filtered or unexported methods
}
FixedBlockAllocator is an Allocator implementation which reduces cgo.Malloc and cgo.Free traffic by allocating entire pages of data at once, and returning pre-assigned block pointers in response to Malloc calls. Malloc calls requesting data buffers larger than the pre-assigned block size will panic. The Free method does not call C.free- instead, the block pointer is simply returned to the allocator to be reused at a later time.
Because the FixedBlockAllocator deals with equally-sized blocks, there is no risk of memory fragmentation. Whenever a Malloc is requested, but there are no free block pointers, a new page will be allocated. Whenever a Free is requested, and post-free the page has no assigned block pointers, and fewer than 1/4 of all block pointers are assigned, the page will be freed. Otherwise, Malloc and Free calls made to this Allocator will simply shuffle around block pointers with no cgo interaction at all.
func CreateFixedBlockAllocator ¶
func CreateFixedBlockAllocator(inner Allocator, pageSize, blockSize, alignment uintptr) (FixedBlockAllocator, error)
CreateFixedBlockAllocator creates a new FixedBlockAllocator with the provided properties. inner - Pages are created using this Allocator pageSize - The size of allocated pages, in bytes. Must be a multiple of blockSize. blockSize - The maximum buffer size of requested allocations. Must be a multiple of alignment. alignment - All block pointers will be along this byte alignment.
Source Files