caramlstore

package

v0.9.2 Latest Latest Go to latest Published: Sep 24, 2023 License: Apache-2.0 Imports: 22 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/raystack/meteor

Links

Open Source Insights

README ¶

caramlstore

Extractor for Machine Learning(ML) Features from caraml-store.

The extractor uses the CoreService.proto service exposed by caraml-store to extract feature tables.

Usage

source:
  name: caramlstore
  scope: caramlstore-stg
  config:
    url: staging.caraml-store.com:80
    max_size_in_mb: 10
    request_timeout: 30s

Inputs

Key	Value	Example	Description	Required?
`url`	`string`	`caraml-store.com:80`	caraml-store's host URL	✅
`max_size_in_mb`	`int`	`10`	Max MB for gRPC client to receive message. Default is 45.	✘
`request_timeout`	`string`	`10s`	Timeout for gRPC requests to caraml-store. Default is 10s.	✘

Outputs

The feature tables are mapped to an Asset with model specific metadata stored using FeatureTable. Please refer the proto definitions for more information.

Field	Value	Sample Value
`resource.urn`	`urn:caramlstore:{scope}:feature_table:{feature_table.project}.{feature_table.name}`	`urn:caramlstore:caramlstore-stg:feature_table:my_project.merchant_uuid_t2_discovery`
`resource.name`	`{feature_table.name}`	`merchant_uuid_t2_discovery`
`resource.service`	`caramlstore`	`caramlstore`
`resource.type`	`feature_table`	`feature_table`
`namespace`	`{feature_table.project}`	`my_project`
`entities`	`[]Entity`
`features`	`[]Feature`
`create_time`	`{feature_table.created_timestamp}`	`2022-08-08T03:17:54Z`
`update_time`	`{feature_table.updated_timestamp}`	`2022-08-08T03:57:54Z`

Entity

Field	Value	Sample Value
`name`	`{entity.name}`	`service_type`
`labels`	`map[string]string`	`{"description":"merchant uuid","value_type":"STRING"}`

Feature

Field	Value	Sample Value
`name`	`{feature.name}`	`avg_gmv_merchant_1d`
`data_type`	`{features.value_type}`	`INT64`

Contributing

Refer to the contribution guidelines for information on contributing to this module.

Documentation ¶

Index ¶

type Client
type Config
type Extractor
- func New(logger log.Logger, client Client) *Extractor

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

This section is empty.

Types ¶

type Client ¶

type Client interface {
	Connect(ctx context.Context, host string, maxSizeInMB int, timeout time.Duration) error
	Projects(ctx context.Context) ([]string, error)
	Entities(ctx context.Context, project string) (map[string]*core.Entity, error)
	FeatureTables(ctx context.Context, project string) ([]*core.FeatureTable, error)
	Close() error
}

type Config ¶

type Config struct {
	URL            string        `json:"url" yaml:"url" mapstructure:"url" validate:"required"`
	MaxSizeInMB    int           `json:"max_size_in_mb" yaml:"max_size_in_mb" mapstructure:"max_size_in_mb"`
	RequestTimeout time.Duration `json:"request_timeout" yaml:"request_timeout" mapstructure:"request_timeout" validate:"min=1ms" default:"10s"`
}

Config holds the set of configuration for the CaraML Store extractor

type Extractor ¶

type Extractor struct {
	plugins.BaseExtractor
	// contains filtered or unexported fields
}

Extractor manages the communication with the CaraML Store service

func New ¶

func New(logger log.Logger, client Client) *Extractor

func (*Extractor) Close ¶ added in v0.9.1

func (e *Extractor) Close() error

func (*Extractor) Extract ¶

func (e *Extractor) Extract(ctx context.Context, emit plugins.Emit) error

Extract checks if the table is valid and extracts the table schema

func (*Extractor) Init ¶

func (e *Extractor) Init(ctx context.Context, config plugins.Config) error

Init initializes the extractor

Source Files ¶

View all Source files

Directories ¶

Path	Synopsis
internal
core
mocks

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL