libgoffi

= libgoffi - idiomatic libffi library and arbitrary function adapter for Go
Christoph Engelbert <https://github.com/noctarius[@noctarius2k]>
clevabit GmbH <https://github.com/clevabit[@clevabit]>
// Settings:
:compat-mode!:
:idseperator: -
// Aliases:
:project-name: libgoffi
:project-handle: libgoffi
:toc:

== TLDR;

[source,go]
----
import (
  goffi "github.com/clevabit/libgoffi"
)

// Import libc library
library := goffi.NewLibrary("libc", goffi.BindNow|goffi.BindGlobal)

// Make a function definition matching the native function's signature
type getpid = func() int

// Create a Go variable of the function type
var fn getpid

// Import the getpid function and map it to the target variable
if err := library.Import("getpid", &fn); err != nil {
  // error handling
}

// Execute the function like it was a standard Go function
println(fmt.sprintf("pid: %d", fn()))
----

== Introduction to libgoffi

image:https://travis-ci.org/clevabit/libgoffi.svg?branch=master[link=https://travis-ci.org/clevabit/libgoffi, window="_blank"]
image:https://ci.appveyor.com/api/projects/status/github/clevabit/libgoffi?branch=master&svg=true[link=https://ci.appveyor.com/project/noctarius/libgoffi, window="_blank"]
image:https://api.codacy.com/project/badge/Grade/0b6a6380bfdf497eb91bd3ea17c8b6ad["Codacy code quality", link="https://www.codacy.com/app/noctarius/libgoffi?utm_source=github.com&utm_medium=referral&utm_content=clevabit/libgoffi&utm_campaign=Badge_Grade"]
image:https://godoc.org/github.com/clevabit/libgoffi?status.svg[link=https://godoc.org/github.com/clevabit/libgoffi, window="_blank"]
image:https://goreportcard.com/badge/github.com/clevabit/libgoffi[link=https://goreportcard.com/report/github.com/clevabit/libgoffi, window="_blank"]
image:https://coveralls.io/repos/github/clevabit/libgoffi/badge.svg?branch=master[link=https://coveralls.io/github/clevabit/libgoffi?branch=master, window="_blank"]

link:http://sourceware.org/libffi/[libffi] is one of the most commonly used libraries
to implement foreign function interfaces in programming languages or frameworks.

While Go has an amazing built-in functionality to interoperate with C functions, the
usage of the CGO pseudo interfaces is not very intuitive. Especially in combination
with dynamic libraries (_dlopen_, _dlsym_, _dlclose_) the complexity increases
exponentially, due to the  of loading and unloading the library and manually determining
the symbol address.

libgoffi implements a wrapper library to automatically load dynamic libraries into the
applicationÄs memory space (using link:https://github.com/achille-roussel/go-dl[go-dl]
by link:https://github.com/achille-roussel[Achille Roussel]), and map imported functions
onto their Go function type counterparts, using libffi's common foreign function interface.

The necessary functions are generated at runtime by utilizing Go's reflection library
and setting up the types and stub according to the given function signatures.

**Attention:** Generating the code statically may be possible, but is not implemented at
time of writing.

libgoffi automatically maps the most commonly used data types between Go and C
bi-directionally.

**Attention:** libgoffi may run with Go compiler versions lower than 1.12, but this is
not tested. Pull requests to support older and newer versions are welcome though.

== Supported Data Types

libgoffi currently supports mapping of the most commonly used Go data types. Mapping can
be applied manually, by specifying both parameter signatures, or automaticly by providing
the Go function signature when importing the function, and have libgoffi figure out the
according C function signature based on the table below.

Mapping for the following data types is implemented:

.Data Type Mapping
|===
| Go Data Type | C Data Type | libffi Data Type

| uint | uint16_t / uint32_t | ffi_type_uint16 / ffi_type_uint32
| uint8 | uint8_t | ffi_type_uint8
| uint16 | uint16_t | ffi_type_uint16
| uint32 | uint32_t | ffi_type_uint32
| uint64 | uint64_t | ffi_type_uint64
| int8 | int8_t | ffi_type_sint8
| int16 | int16_t | ffi_type_sint16
| int32 | int32_t | ffi_type_sint32
| int64 | int64_t | ffi_type_sint64
| float32 | float_t | ffi_type_float
| float64 | double_t | ffi_type_double
| unsafe.Pointer | void * | ffi_type_pointer
| uintptr | void * | ffi_type_pointer
| safe pointers (&var) | void * | ffi_type_pointer
| - | void | ffi_type_void
|===

**Attention:** The Go types _uint_ and _int_ mapping is platform specific and
determined by looking at the maximum value of a signed int in C. Please be aware
when automatically mapping those data types. It is advised to use more specific
data types, such as int32 or uint32 to be platform independent.

**Attention:** Mapping of structs is not supported. It is recommended to create
all pointers to structs, that need to be passed to C code, by allocating them
using _C.malloc(…)_. This prevents the Go runtime to throw the
"_Go Pointer to Go Pointer_" exception.

**Attention:** When passing a Go String to a function, remember, that it is mapped to
a _char *_ data type in C. That means, the string will be extended by adding _0x00_
byte (a zero byte, _\0_) at the end. Therefore, if the function in questions requires
passing the length of the string, the actual string passed is one byte longer than the
string in Go (_len(msg)+1_).

**Attention:** Returning a _char *_ from a function in C (which is uncommon), libgoffi
will normally map it to a string in Go. Furthermore, libgoffi will immediately free the
actual C pointer returned from the function. If this behavior is unintentional, the
mapping must specifically ask to return the _char *_ itself. The transformation into a
Go String (using _C.GoString(ptr)_) must be done manually, as freeing of the pointer.

== Supported Operating Systems

lobgoffi should support almost every posix compliant OS, however, only code for Linux and
OSX (Darwin) has been tested.

A few lines of code should make it possible to support operating systems like
FreeBSD or similar. Pull requests to add support for other operating systems are
welcome.

**Windows** is not supported, since Windows does not support dlopen, dlsym and dlclose.
It should be totally possible to support the WinAPI functions to load .DLL files though.
Again pull requests adding support for Windows are welcome.

== Not Supported

* lobgoffi does not support signature checking. When executing an imported function using
a wrong function signature, your program may just return a wrong value. In the worst
case, however, it may actually crash with a segmentation fault or another exception
of any kind. In any case, it will not behave as expected (or does it? :-)).

* As mentioned above, Windows is also not supported, as are any other posix compliant
operating systems other than Linux and OSX (Darwin). In theory any posix OS
supported by both Go and libffi should be possible to support though.

* Mapping of Go structs is not supported and not the subject of this library adapter. Pointers
to Go memory are always complicated to handle, and error prone. More information on CGO
interaction and Go pointers can be found in the
link:https://golang.org/cmd/cgo/#hdr-Passing_pointers[official Go documentation].

* Last but not least, function pointers are not officially supported or tested, but may
work when exported as C functions. This might be officially supported in the future though.

== Loading a Library

Thanks to the _go-dl_ library which is used to load the underlying dynamic library,
libgoffi tries hard to automatically determine the actual path of a library.

Loading a library is normally as easy as asking by its name:

[source,go]
----
import (
  goffi "github.com/clevabit/libgoffi"
)

library := goffi.NewLibrary("libc", goffi.BindNow|goffi.BindGlobal)
----

libgoffi provides some binding flags of the posix API, more specifically:

* BindLazy
* BindNow
* BindLocal
* BindGlobal

The binding flags are XOR'ed together before being passed to the loader.

More information on those flags can be found in the
link:https://linux.die.net/man/3/dlopen[Linux manpages].

== Import Functions

Importing functions from the loaded library is provided using 3 different styles,
depending on how much type mapping is necessary and how complex function types
are designed.

=== Fully Automatic Data Type Mapping

libgoffi is able to provide a fully automatic type mapping, which is probably
enough to map the most common functions.

The following example expects the _libc_ library to already being loaded into
the application as shown in the previous section.

[source,go]
----
// Make a function definition matching the native function's signature
type getpid = func() int

// Create a Go variable of the function type
var fn getpid

// Import the getpid function and map it to the target variable
if err := library.Import("getpid", &fn); err != nil {
  // error handling
}

// Execute the function like it was a standard Go function
println(fmt.sprintf("pid: %d", fn()))
----

In this example we imported the _getpid_ function from libc, which in itself returns
the pid (process identifier) of the currently running application, that said, our
demo application.

This mapping type also works for functions that expect one or more parameters.

[source,go]
----
type sqrt = func(float64) float64

var fn sqrt
if err := library.Import("sqrt", &fn); err != nil {
  // error handling
}
println(fmt.sprintf("sqrt of 9.0: %f", fn(9.)))
----

It is also always possible to map out error return types as the last parameter of the
function definition. The error will not be mapped out to the C function signature, but used
by the library to report errors during execution of the function, like illegal parameter
values.

An example of such a function mapping would be (using the sqrt example again):

[source,go]
----
type sqrt = func(float64) (float64, error)

var fn sqrt
if err := library.Import("sqrt", &fn); err != nil {
  // error handling
}
sq, err := fn(9.)
if err != nil {
  // error handling
}
println(fmt.sprintf("sqrt of 9.0: %f", sq))
----

If no explicit error handling is requested as part of the function's signature, libgoffi
will use panics to report the malfunctioning behavior. It is advised to explicitly map
errors are return parameters to prevent unexpected panics.

=== New Function Import

In addition to mapping a C function to an existing variable of a specific Go function
type, libgoffi can also create function mappers for freely defined (reflective)
function definitions.

For example we can import both of the above functions again, but this time using
the explicit factory function.

[source,go]
----
// Create a new function which returns an int and an error (the third parameter)
fn, err := library.NewImport("getpid", goffi.TypeInt, true)
if err != nil {
  // error handling
}

// Type assertion to the specific Go function type
getpid, ok := fn.(func()(int, error))
if !ok {
  // error handling
}

// Execute the function like it was a standard Go function
println(fmt.sprintf("pid: %d", getpid()))
----

In this example we mapped the _getpid_ function again and told the mapper we also want
to report errors back. Remember, not reporting errors may result in a runtime panic
in case of any issues with the mapping.

To map the returned function to a callable variable, a type assertions is required.
Type assertions provide the benefit of automatic runtime type checking.

For the next example we import the _sqrt_ function again, but this time we will not map
out errors though (third parameter is false). However, we also provide the parameter
signature (a single float64 parameter). The parameter signature is a variadic argument
and can take an arbitrary number of type arguments.

[source,go]
----
fn, err := library.NewImport("sqrt", goffi.TypeFloat64, false, goffi.TypeFloat64)
if err != nil {
  // error handling
}

sqrt, ok := fn.(func(float64) float64)
if !ok {
  // error handling
}

println(fmt.sprintf("sqrt of 9.0: %f", sqrt(9.)))
----

=== Complex Data Type Mapping

Sometimes, a more complex type mapping is necessary. This is especially the
case, when the there is no automatic mapping for a library specific C data type.

libgoffi provides a specific import function for complex use cases. It is able to
be provided with a specific set of Go and C side function type definitions.

libgoffi will try its best to map the given C type to the Go type, and vise versa.
It can, for example, be used to map number types in C or Go to another data type in
the other language. Complex mapping of numbers can be dangerous though and types may
be incorrectly be narrowed or widened if erroneously specificied. Also be aware of
potential overflow handling when mapping between unsigned and signed data types.

To show a more complex example, we will be passing an int to the _sqrt_ function, even
though the C function clearly expects a float parameter. We also ask to return an int.
The return value translation is possible but will truncate the data to an integer
representation.

The type translation is automatically handled by libgoffi before passing the value
onwards to the imported function.

It is possible to just translate the parameter or return type, too.

[source, go]
----
// Define the Go function signature
fnGo := reflect.FuncOf(
  []reflect.Type{goffi.TypeInt},     // input types
  []reflect.Type{goffi.TypeInt},     // output types
  false,                             // non-variadic
)

// Define the C function signature
fnC := reflect.FuncOf(
  []reflect.Type{goffi.TypeFloat64}, // input types
  []reflect.Type{goffi.TypeFloat64}, // output types
  false,                             // non-variadic
)

// Import the function
fn, err := library.NewImportComplex("sqrt", fnGo, fnC)
if err != nil {
  // error handling
}

sqrt, ok := fn.(func(int) int)
if !ok {
  // error handling
}

println(fmt.sprintf("sqrt of 9: %d", sqrt(9)))
----

== Closing a Loaded Library

libgoffi uses internal caches to store state and loaded symbols. Furthermore, it also
allocates memory outside of the Go heap. That said, a loaded library should be closed
explicitly to free allocated resources.

A simple call to the _Close()_ function is enough.

[source,go]
----
if err := library.Close(); err != nil {
  // error handling
}
----

== Why libgoffi?

libgoffi provides Go idiomatic loading, importing and mapping of C functions, without
the complexity of manual handling the CGO, when dealing with dynamic libraries.

While Go and CGO provide a good solution to support statically linked libraries, depending
on the use case, linking all libraries statically may not be the preferred solution.

Especially in embedded environments space is limitted and a library already on disk (and
used by other tools) doesn't need to be duplicated with static linking.

Loading a dynamic library, importing symbols and mapping calls, however, can be a tedious
job. That's why lobgoffi hides the complexity, and provides a clean and idiomatic Go
interface.

== License

libgoffi is provided under the Apache License 2.0. That means, it can freely be copied,
used, updated, changed. Code changes do not need to be upstreamed back to the project,
we'd love however to see users to provide additional functionality, mappings or just
bugfixes or feature requests and ideas.

== Liability

libgoffi is provided by the link:https://www.clevabit.com[clevabit GmbH] for free and
as is. clevabit is not liable for any damage on software, hardware, or of any other
nature, which is related to the usage of this library.