lib

package module
v0.0.0-...-0288511 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 23, 2020 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExtractReceivedInfo

func ExtractReceivedInfo(r *http.Request, pullInfo *PullInfo) error

ExtractReceivedInfo extracts the information received through the HTTP request body needed for the pull configuration. An error is returned if any errors occur during the function execution.

func FileName

func FileName(info StorageInfo) string

FileName constructs a name of file in which the message will be written. Returned value is a string which consists of current date and hour, followed by chosen prefix, message ID and file extension.

func PersistData

func PersistData(ctx context.Context, data []byte, info StorageInfo) error

PersistData stores a message in GCS bucket. Before storing, the unique file is created using information from storage configuration. Each message is written in a separate file. Returned result is an error which defines the validity of the function action.

func Pull

func Pull(ctx context.Context, info *PullInfo, storageInfo StorageInfo, subConf *SubConf) error

Pull function pulls messages from provided Pub/Sub subscription and calls storing function on each pulled message. As a part of a process, Pull creates a client that will receive blocks of messages. Received blocks will be of a limited size if synchronous option is enabled. Synchronous pull stores fixed number of messages and cancels the context which prevents further receiving. If the streaming pull option is chosen, the client receives blocks of a variable sizes until context duration expires. An error is returned if any errors occur during the function execution.

func ResourceCloser

func ResourceCloser(client *storage.Client, writer *storage.Writer) error

ResourceCloser closes the client and writer components of the GCP storage service. Returned result is an error which defines the validity of the function action.

func SetInvokerInfo

func SetInvokerInfo(invokerInfo *InvokerInfo) error

SetInvokerInfo sets the parameters of a invoker configuration by extracting values ​​from the corresponding environment variables. An error is returned if any errors occur during the function execution.

func SetPullInfo

func SetPullInfo(pullInfo *PullInfo) error

SetPullInfo sets the values of the pull configuration by extracting the values ​​from the corresponding environment variables. An error is returned if any error occurs during the function execution.

func SetStorageInfo

func SetStorageInfo(storageInfo *StorageInfo) error

SetStorageInfo sets the parameters of a storage config. An error is returned if any errors occur during the function execution.

func SetSubscriberConf

func SetSubscriberConf(subscriberConf *SubConf, synchronous bool) error

SetSubscriberConf sets the parameters of a subscriber configuration by extracting values ​​from the corresponding environment variables. The synchronous and maxExtension parameters depend on the type of pull which is determined by passed bool variable. An error is returned if any errors occur during the function execution.

Types

type InvokerInfo

type InvokerInfo struct {
	NumberOfMessages  string //number of messages pull function will pull and store (only for synchronous pull)
	NumberOfSeconds   int    //time duration of a context for which the messages will be pulled
	NumberOfInstances int    //number of pull function instances that will run in parallel
	InstanceNumber    int    //help parameter used for logging error messages (indicates on which instance the error occurred)
	FunctionURL       string //URL of a Cloud function which will be triggered by invoker
}

InvokerInfo represents a invoker configuration. It holds information needed for invocation of pull functions.

type PullInfo

type PullInfo struct {
	ProjectID        string // the ID of a project in which the topic is located
	SubID            string // the ID of a subscription the messages will be pulled from
	NumberOfMessages int    // the number of messages pull function will persist in one call (this applies only to the synchronous version)
	NumberOfSeconds  int    // the time duration in which the messages will be received
}

PullInfo represents pull configuration. It holds information needed for Pull function.

type StorageInfo

type StorageInfo struct {
	MessageID string // ID of a message (used for naming a file in which the message will be written)
	BucketID  string // ID of a bucket in which messages will be stored
	Prefix    string // prefix of a file name
	Extension string // file extension (txt, json, yaml, etc.)
}

StorageInfo represents storage configuration. It holds information needed for storing messages to GCS.

type SubConf

type SubConf struct {
	Synchronous            bool
	MaxExtension           int
	MaxOutstandingMessages int
	MaxOutstandingBytes    int
	NumOfGoroutines        int
}

SubConf represents subscriber configuration. It holds receive information.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL