Documentation ¶
Overview ¶
go-rknnlite provides Go language bindings for the RKNN Toolkit2 C API interface. It aims to provide lite bindings in the spirit of the closed source Python lite bindings used for running AI Inference models on the Rockchip NPU via the RKNN software stack.
These bindings have only been tested on the RK3588 (specifically the Radxa Rock Pi 5B) but should work on other RK3588 based SBC's. It should also work with other models in the RK35xx series supported by the RKNN Toolkit2.
See example code and usage in the examples subdirectory.
Index ¶
- Constants
- func GetTop(pfProb []float32, pfMaxProb []float32, pMaxClass []int32, outputCount int32, ...) int
- type AttrMaxDimensions
- type CoreMask
- type ErrorCodes
- type IONumber
- type Input
- type Output
- type Pool
- type Probability
- type Runtime
- func (r *Runtime) Close() error
- func (r *Runtime) GetOutputs(nOutputs uint32) ([]Output, error)
- func (r *Runtime) Inference(mats []gocv.Mat) ([]Output, error)
- func (r *Runtime) QueryInputTensors() ([]TensorAttr, error)
- func (r *Runtime) QueryModelIONumber() (ioNum IONumber, err error)
- func (r *Runtime) QueryOutputTensors() ([]TensorAttr, error)
- func (r *Runtime) RunModel() error
- func (r *Runtime) SDKVersion() (SDKVersion, error)
- func (r *Runtime) SetInputs(inputs []Input) error
- type SDKVersion
- type TensorAttr
- type TensorFormat
- type TensorQntType
- type TensorType
Constants ¶
const MAX_TOP_NUM = 20
Variables ¶
This section is empty.
Functions ¶
Types ¶
type AttrMaxDimensions ¶
type AttrMaxDimensions int
AttrMaxDimensions are the maximum dimensions for an attribute in a tensor
const ( AttrMaxDimension AttrMaxDimensions = C.RKNN_MAX_DIMS AttrMaxChannels AttrMaxDimensions = C.RKNN_MAX_NUM_CHANNEL AttrMaxNameLength AttrMaxDimensions = C.RKNN_MAX_NAME_LEN AttrMaxDynShape AttrMaxDimensions = C.RKNN_MAX_DYNAMIC_SHAPE_NUM )
maximum field lengths of attributes in a tensor
type CoreMask ¶
type CoreMask int
CoreMask wraps C.rknn_core_mask
const ( NPUCoreAuto CoreMask = C.RKNN_NPU_CORE_AUTO NPUCore0 CoreMask = C.RKNN_NPU_CORE_0 NPUCore1 CoreMask = C.RKNN_NPU_CORE_1 NPUCore2 CoreMask = C.RKNN_NPU_CORE_2 NPUCore01 CoreMask = C.RKNN_NPU_CORE_0_1 NPUCore012 CoreMask = C.RKNN_NPU_CORE_0_1_2 )
rknn_core_mask values used to target which cores on the NPU the model is run on. The rk3588 has three cores, auto will pick an idle core to run the model on, whilst the others specify the specific core or combined number of cores to run. For multi-core modes the following ops have better acceleration: Conv, DepthwiseConvolution, Add, Concat, Relu, Clip, Relu6, ThresholdedRelu, Prelu, and LeakyRelu. Other type of ops will fallback to Core0 to continue running
type ErrorCodes ¶
type ErrorCodes int
ErrorCodes
const ( Success ErrorCodes = C.RKNN_SUCC ErrFail ErrorCodes = C.RKNN_ERR_FAIL ErrTimeout ErrorCodes = C.RKNN_ERR_TIMEOUT ErrMallocFail ErrorCodes = C.RKNN_ERR_MALLOC_FAIL ErrParamInvalid ErrorCodes = C.RKNN_ERR_PARAM_INVALID ErrModelInvalid ErrorCodes = C.RKNN_ERR_MODEL_INVALID ErrCtxInvalid ErrorCodes = C.RKNN_ERR_CTX_INVALID ErrInputInvalid ErrorCodes = C.RKNN_ERR_INPUT_INVALID ErrOutputInvalid ErrorCodes = C.RKNN_ERR_OUTPUT_INVALID ErrDeviceMismatch ErrorCodes = C.RKNN_ERR_DEVICE_UNMATCH ErrPreCompiledModel ErrorCodes = C.RKNN_ERR_INCOMPATILE_PRE_COMPILE_MODEL ErrOptimizationVersion ErrorCodes = C.RKNN_ERR_INCOMPATILE_OPTIMIZATION_LEVEL_VERSION ErrPlatformMismatch ErrorCodes = C.RKNN_ERR_TARGET_PLATFORM_UNMATCH )
error code values returned by the C API
func (ErrorCodes) String ¶
func (e ErrorCodes) String() string
String returns a readable description of the error code
type Input ¶
type Input struct { // Index is the input index Index uint32 // Buf is the gocv Mat input Buf unsafe.Pointer // Size is the number of bytes of Buf Size uint32 // Passthrough defines the mode, if True the buf data is passed directly to // the input node of the rknn model without any conversion. If False the // buf data is converted into an input consistent with the model according // to the following type and fmt PassThrough bool // Type is the data type of Buf. This is a required parameter if Passthrough // is False Type TensorType // Fmt is the data format of Buf. This is a required parameter if Passthrough // is False Fmt TensorFormat }
Input represents the C.rknn_input struct and defines the Input used for inference
type Output ¶
type Output struct { WantFloat uint8 // want transfer output data to float IsPrealloc uint8 // whether buf is pre-allocated Index uint32 // the output index Buf []float32 // the output buf Size uint32 // the size of output buf }
Output wraps C.rknn_output
type Pool ¶
type Pool struct {
// contains filtered or unexported fields
}
Pool is a simple runtime pool to open multiple of the same Model across all NPU cores
type Probability ¶
func GetTop5 ¶
func GetTop5(outputs []Output) []Probability
GetTop5 outputs the Top5 matches in the model, with left column as label index and right column the match probability. The results are returned in the Probability slice in descending order from top match.
type Runtime ¶
type Runtime struct {
// contains filtered or unexported fields
}
Runtime defines the RKNN run time instance
func NewRuntime ¶
NewRuntime returns a RKNN run time instance. Provide the full path and filename of the RKNN compiled model file to run.
func (*Runtime) Close ¶
Close wraps C.rknn_destroy which unloads the RKNN model from the runtime and destroys the context releasing all C resources
func (*Runtime) GetOutputs ¶
GetOutputs returns the Output results
func (*Runtime) QueryInputTensors ¶
func (r *Runtime) QueryInputTensors() ([]TensorAttr, error)
QueryInputTensors gets the model Input Tensor attributes
func (*Runtime) QueryModelIONumber ¶
QueryModelIONumber queries the number of Input and Output tensors of the model
func (*Runtime) QueryOutputTensors ¶
func (r *Runtime) QueryOutputTensors() ([]TensorAttr, error)
QueryOutputTensors gets the model Output Tensor attributes
func (*Runtime) SDKVersion ¶
func (r *Runtime) SDKVersion() (SDKVersion, error)
SDKVersion returns the RKNN API and Driver versions
type SDKVersion ¶
SDKVersion represents the C.rknn_sdk_version struct
type TensorAttr ¶
type TensorAttr struct { Index uint32 NDims uint32 Dims [AttrMaxDimension]uint32 Name string NElems uint32 Size uint32 Fmt TensorFormat Type TensorType QntType TensorQntType FL int8 ZP int32 Scale float32 WStride uint32 SizeWithStride uint32 PassThrough bool HStride uint32 }
TensorAttr represents the C.rknn_tensor_attr structure
func (TensorAttr) String ¶
func (a TensorAttr) String() string
String returns the TensorAttr's attributes formatted as a string
type TensorFormat ¶
type TensorFormat int
TensorFormat wraps C.rknn_tensor_format
const ( TensorNCHW TensorFormat = C.RKNN_TENSOR_NCHW TensorNHWC TensorFormat = C.RKNN_TENSOR_NHWC TensorNC1HWC2 TensorFormat = C.RKNN_TENSOR_NC1HWC2 TensorUndefined TensorFormat = C.RKNN_TENSOR_UNDEFINED )
func (TensorFormat) String ¶
func (t TensorFormat) String() string
String returns a readable description of the TensorFormat
type TensorQntType ¶
type TensorQntType int
TensorQntType wraps C.rknn_tensor_qnt_type
const ( TensorQntNone TensorQntType = C.RKNN_TENSOR_QNT_NONE TensorQntDFP TensorQntType = C.RKNN_TENSOR_QNT_DFP TensorQntAffine TensorQntType = C.RKNN_TENSOR_QNT_AFFINE_ASYMMETRIC )
func (TensorQntType) String ¶
func (t TensorQntType) String() string
String returns a readable description of the TensorQntType
type TensorType ¶
type TensorType int
TensorType wraps C.rknn_tensor_type
const ( TensorFloat32 TensorType = C.RKNN_TENSOR_FLOAT32 TensorFloat16 TensorType = C.RKNN_TENSOR_FLOAT16 TensorInt8 TensorType = C.RKNN_TENSOR_INT8 TensorUint8 TensorType = C.RKNN_TENSOR_UINT8 TensorInt16 TensorType = C.RKNN_TENSOR_INT16 TensorUint16 TensorType = C.RKNN_TENSOR_UINT16 TensorInt32 TensorType = C.RKNN_TENSOR_INT32 TensorUint32 TensorType = C.RKNN_TENSOR_UINT32 TensorInt64 TensorType = C.RKNN_TENSOR_INT64 TensorBool TensorType = C.RKNN_TENSOR_BOOL TensorInt4 TensorType = C.RKNN_TENSOR_INT4 )
func (TensorType) String ¶
func (t TensorType) String() string
String returns a readable description of the TensorType
Directories ¶
Path | Synopsis |
---|---|
example
|
|
mobilenet
Example code showing how to perform inferencing using a MobileNetv1 model.
|
Example code showing how to perform inferencing using a MobileNetv1 model. |
pool
Running multiple Runtimes in a Pool allows you to take advantage of all three NPU cores to significantly reduce average inferencing time.
|
Running multiple Runtimes in a Pool allows you to take advantage of all three NPU cores to significantly reduce average inferencing time. |