gonnx

package module
v0.0.0-...-c9ed6d8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 29, 2023 License: AGPL-3.0 Imports: 8 Imported by: 0

README

GONNX

This library allows you to use ONNX Runtime models in Go.

Examples

See examples for more details.

License

This code was initially cloned from: https://github.com/peer-ai-org/onnxruntime_go.git and is subject to the terms of the MIT License. The above repository initially obtained the code from: https://github.com/yalue/onnxruntime_go.git. Please refer to the original repositories for the full license text.

Code that does not have the above disclamer, was written by me and is subject to the terms of the AGPLv3 License. Please refer to the LICENSE file for the full license text.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var NotInitializedError error = fmt.Errorf("InitializeRuntime() has either " +
	"not yet been called, or did not return successfully")

Functions

func CheckInputsOutputs 

func CheckInputsOutputs(inputs []*TensorWithType, outputs []*TensorWithType, inputNames, outputNames []string) error

func ConvertNames 

func ConvertNames(names []string) []*C.char

func ConvertTensors 

func ConvertTensors(tensors []*TensorWithType) []*C.OrtValue

func DestroyEnvironment 

func DestroyEnvironment() error

Call this function to cleanup the internal onnxruntime environment when it is no longer needed.

func GetTensorElementDataType 

func GetTensorElementDataType[T TensorData]() C.ONNXTensorElementDataType

Returns the ONNX enum value used to indicate TensorData type T.

func InitializeEnvironment 

func InitializeEnvironment() error

Call this function to initialize the internal onnxruntime environment. If this doesn't return an error, the caller will be responsible for calling DestroyEnvironment to free the onnxruntime state when no longer needed.

func IsInitialized 

func IsInitialized() bool

Returns false if the onnxruntime package is not initialized. Called internally by several functions, to avoid segfaulting if InitializeEnvironment hasn't been called yet.

func NewEmptyTensorWithType 

func NewEmptyTensorWithType(tensorType string, s Shape) (interface{}, error)

func RandomSelect 

func RandomSelect(probabilities []float64) int

func Reshape 

func Reshape(data []float64, shape []int64) [][]float64

func SetSharedLibraryPath 

func SetSharedLibraryPath(path string)

Use this function to set the path to the "onnxruntime.so" or "onnxruntime.dll" function. By default, it will be set to "onnxruntime.so" on non-Windows systems, and "onnxruntime.dll" on Windows. Users wishing to specify a particular location of this library must call this function prior to calling onnxruntime.InitializeEnvironment().

func Softmax 

func Softmax(x []float64) []float64

function to calculate softmax

func TakeTopP 

func TakeTopP(logs []float64, topP float64) ([]float64, []int)

Types

type FloatData 

type FloatData interface {
	~float32 | ~float64
}

type IntData 

type IntData interface {
	~int8 | ~uint8 | ~int16 | ~uint16 | ~int32 | ~uint32 | ~int64 | ~uint64
}

type Log 

type Log struct {
	Value float64
	Index int
}

type RunV3GenOptions 

type RunV3GenOptions struct {
	MaxTokens           int
	MaxNewTokens        int
	TopP                float64
	Temperature         float64
	EOSTokenID          int
	ReplacementIndexes  []int
	AttentionMaskIndex  int
	UseCacheBranchIndex int
	SeparateDecoder     bool
}

type SessionV3 

type SessionV3 struct {
	// contains filtered or unexported fields
}

func NewSessionV3 

func NewSessionV3(path string, opts ...string) (*SessionV3, error)

func (*SessionV3) Destroy 

func (s *SessionV3) Destroy() error

func (*SessionV3) GetInputNames 

func (s *SessionV3) GetInputNames() []string

func (*SessionV3) GetInputShapes 

func (s *SessionV3) GetInputShapes() (shapeTypes []ShapeType)

func (*SessionV3) GetInputTypes 

func (s *SessionV3) GetInputTypes() []string

func (*SessionV3) GetOutputNames 

func (s *SessionV3) GetOutputNames() []string

func (*SessionV3) GetOutputShapes 

func (s *SessionV3) GetOutputShapes() (shapeTypes []ShapeType)

func (*SessionV3) GetOutputTypes 

func (s *SessionV3) GetOutputTypes() []string

func (*SessionV3) Run 

func (s *SessionV3) Run(inputs []*TensorWithType) (outputs []*TensorWithType, err error)

func (*SessionV3) RunDecoder 

func (s *SessionV3) RunDecoder(inputs []*TensorWithType, opt *RunV3GenOptions) (outTokenIds []int64, err error)

func (*SessionV3) RunGen 

func (s *SessionV3) RunGen(inputs []*TensorWithType, opt *RunV3GenOptions) (outputs []*TensorWithType, err error)

func (*SessionV3) RunMergedDecoder 

func (s *SessionV3) RunMergedDecoder(inputs []*TensorWithType, opt *RunV3GenOptions) (outTokenIds []int64, err error)

type Shape 

type Shape []int64

The Shape type holds the shape of the tensors used by the network input and outputs.

func NewShape 

func NewShape(dimensions ...int64) Shape

Returns a Shape, with the given dimensions.

func (Shape) Clone 

func (s Shape) Clone() Shape

Makes and returns a deep copy of the Shape.

func (Shape) FlattenedSize 

func (s Shape) FlattenedSize() int64

Returns the total number of elements in a tensor with the given shape.

func (Shape) String 

func (s Shape) String() string

type ShapeType 

type ShapeType struct {
	Shape         []int64
	SymbolicShape []string
	Type          string
}

type Tensor 

type Tensor[T TensorData] struct {
	// contains filtered or unexported fields
}

func GetTensor 

func GetTensor[T TensorData](value *C.OrtValue, elementCount int64, data []T, shape []int64) (*Tensor[T], error)

func NewEmptyTensor 

func NewEmptyTensor[T TensorData](s Shape) (*Tensor[T], error)

Creates a new empty tensor with the given shape. The shape provided to this function is copied, and is no longer needed after this function returns.

func NewTensor 

func NewTensor[T TensorData](s Shape, data []T) (*Tensor[T], error)

Creates a new tensor backed by an existing data slice. The shape provided to this function is copied, and is no longer needed after this function returns. If the data slice is longer than s.FlattenedSize(), then only the first portion of the data will be used.

func (*Tensor[T]) Clone 

func (t *Tensor[T]) Clone() (*Tensor[T], error)

Makes a deep copy of the tensor, including its ONNXRuntime value. The Tensor returned by this function must be destroyed when no longer needed.

func (*Tensor[_]) Destroy 

func (t *Tensor[_]) Destroy() error

Cleans up and frees the memory associated with this tensor.

func (*Tensor[T]) GetData 

func (t *Tensor[T]) GetData() []T

Returns the slice containing the tensor's underlying data. The contents of the slice can be read or written to get or set the tensor's contents.

func (*Tensor[_]) GetShape 

func (t *Tensor[_]) GetShape() Shape

Returns the shape of the tensor. The returned shape is only a copy; modifying this does *not* change the shape of the underlying tensor. (Modifying the tensor's shape can only be accomplished by Destroying and recreating the tensor with the same data.)

type TensorData 

type TensorData interface {
	FloatData | IntData | ~bool
}

This is used as a type constraint for the generic Tensor type.

type TensorWithType 

type TensorWithType struct {
	TensorType string
	Tensor     interface{}
}

func (*TensorWithType) Destroy 

func (t *TensorWithType) Destroy() error

func (*TensorWithType) GetData 

func (t *TensorWithType) GetData() interface{}

func (*TensorWithType) GetShape 

func (t *TensorWithType) GetShape() []int64

Source Files

View all Source files

Directories

Path Synopsis
examples
basic

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.