common

package
v1.4.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 25, 2023 License: UPL-1.0 Imports: 4 Imported by: 0

Documentation

Overview

Package common provides common utilities used for NoSQL client.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type InternalRequestData added in v1.2.2 

type InternalRequestData struct {
	RateLimiterPair
	// contains filtered or unexported fields
}

InternalRequestData is the actual struct that gets included in every Request type

func (*InternalRequestData) GetRetryTime added in v1.2.2 

func (ird *InternalRequestData) GetRetryTime() time.Duration

GetRetryTime returns the current time spent in the client in retries

func (*InternalRequestData) SetRetryTime added in v1.2.2 

func (ird *InternalRequestData) SetRetryTime(d time.Duration)

SetRetryTime sets the current time spent in the client in retries

type InternalRequestDataInt added in v1.2.2 

type InternalRequestDataInt interface {
	RateLimiterPairInt
	GetRetryTime() time.Duration
	SetRetryTime(d time.Duration)
}

InternalRequestDataInt is used to give all requests a set of common internal data (rate limiters, retry stats, etc)

type RateLimiter added in v1.2.2 

type RateLimiter interface {

	// ConsumeUnits consumes a number of units, blocking until the units are available.
	// It returns the amount of time blocked. If not blocked, 0 is returned.
	ConsumeUnits(units int64) time.Duration

	// TryConsumeUnits consumes the specified number of units if they can be returned
	// immediately without waiting.
	// It returns true if the units were consumed, false if they were not immediately available.
	// If units was zero, true return means the limiter is currently below its limit.
	// units can be zero to poll if the limiter is currently over its limit. Pass negative units to
	// "give back" units (same as calling ConsumeUnits with a negative value).
	TryConsumeUnits(units int64) bool

	// ConsumeUnitsWithTimeout attempts to consume a number of units, blocking until the units are
	// available or the specified timeout expires. It returns the amount of time blocked.
	// If not blocked, 0 is returned.
	//
	// units can be a negative value to "give back" units.
	// Pass a timeout value of 0 to block indefinitely. To poll if the limiter is
	// currently over its limit, use TryConsumeUnits() instead.
	// If alwaysConsume is true, consume units even on timeout.
	ConsumeUnitsWithTimeout(units int64, timeout time.Duration, alwaysConsume bool) (time.Duration, error)

	// GetLimitPerSecond returns the number of units configured for this rate limiter instance
	// (the max number of units per second this limiter allows).
	GetLimitPerSecond() float64

	// SetDuration sets the duration for this rate limiter instance.
	// The duration specifies how far back in time the limiter will
	// go to consume previously unused units.
	//
	// For example, if a limiter had a limit of 1000 units and a
	// duration of 5 seconds, and had no units consumed for at least
	// 5 seconds, a call to TryConsumeUnits(5000) will succeed
	// immediately with no waiting.
	SetDuration(durationSecs float64)

	// GetDuration returns the duration in seconds configured for this rate limiter instance
	GetDuration() float64

	// Reset resets the rate limiter as if it was newly constructed.
	Reset()

	// SetLimitPerSecond sets a new limit (units per second) on the limiter.
	//
	// Note that implementing rate limiters should fully support non-integer
	// (float64) values internally, to avoid issues when the limits are set
	// very low.
	// Changing the limit may lead to unexpected spiky behavior, and may
	// affect other goroutines currently operating on the same limiter instance.
	SetLimitPerSecond(rateLimitPerSecond float64)

	// ConsumeUnitsUnconditionally consumes units without checking or waiting.
	// the internal amount of units consumed will be updated by the given
	// amount, regardless of its current over/under limit state.
	// The number of units to consume may be negative to "give back" units.
	ConsumeUnitsUnconditionally(units int64)

	// GetCurrentRate returns the current rate as a percentage of current limit.
	GetCurrentRate() float64

	// SetCurrentRate sets the current rate as a percentage of current limit.
	// This modifies the internal limiter values; it does not modify
	// the rate limit.
	// rateToSet may be greater than 100.0 to set the limiter to "over its limit".
	SetCurrentRate(rateToSet float64)
}

RateLimiter interface provides default methods that all rate limiters must implement.

Thread safety

It is expected that all implementing classes of this interface may be used by multiple goroutines concurrently. For example, many goroutines may be using the same rate limiter instance to ensure that all of them together do not exceed a given limit.

Typical usage

The simplest use of the rate limiter is to consume a number of units, blocking until they are successfully consumed: 

delay := rateLimiter.ConsumeUnits(units)
// delay indicates how long the consume delayed

To poll a limiter to see if it is currently over the limit: 

if rateLimiter.TryConsumeUnits(0) {
  // limiter is below its limit
}

To attempt to consume units, only if they can be immediately consumed without waiting: 

if ratelimiter.TryConsumeUnits(units) {
  // successful consume
} else {
  // could not consume units without waiting
}

Usages with timeouts

In cases where the number of units an operation will consume is already known before the operation, a simple one-shot method can be used: 

var units int64 = (known units the operation will use)
alwaysConsume := false // don't consume if we time out
delay, err := rateLimiter.ConsumeUnitsWithTimeout(units, timeout, alwaysConsume)
if err != nil {
  // we could not do the operation within the given timeframe.
  ...skip operation...
} else {
  // we waited <delay> for the units to be consumed, and the consume was successful
  ...do operation...
}

In cases where the number of units an operation will consume is not known before the operation, typically two rate limiter calls would be used: one to wait till the limiter is below its limit, and a second to update the limiter with used units: 

// wait until we're under the limit
delay, err := rateLimiter.ConsumeUnitsWithTimeout(0, timeout, false)
if err != nil {
  // we could not do the operation within the given timeframe.
  ...skip operation...
}
// we waited <delay> to be under the limit, and were successful
var units int64  = (...do operation, get number of units used...)
// update rate limiter with consumed units. Next operation on this
// limiter may delay due to it being over its limit.
rateLimiter.ConsumeUnitsUnconditionally(units)

Alternately, the operation could be always performed, and then the limiter could try to wait for the units to be consumed: 

var units int64  = (...do operation, get number of units used...)
alwaysConsume := true // consume, even if we time out
delay, err := rateLimiter.ConsumeUnitsWithTimeout(units, timeout, alwaysConsume)
if err != nil {
  // the rate limiter could not consume the units and go
  // below the limit in the given timeframe. Units are
  // consumed anyway, and the next call to this limiter
  // will likely delay
} else {
  // we waited <delay> for the units to be consumed, and the
  // consume was successful
}

Limiter duration

Implementing rate limiters should support a configurable "duration". This is sometimes referred to as a "burst mode", or a "window time", or "burst duration".

This will allow consumes of units that were not consumed in the recent past. For example, if a limiter allows for 100 units per second, and is not used for 5 seconds, it should allow an immediate consume of 500 units with no delay upon a consume call, assuming that the limiter's duration is set to at least 5 seconds.

The maximum length of time for this duration should be configurable. In all cases a limiter should set a reasonable minimum duration, such that a call to TryConsumeUnits(1) has a chance of succeeding. It is up to the limiter implementation to determine if units from the past before the limiter was created or reset are available for use.

type RateLimiterPair added in v1.2.2 

type RateLimiterPair struct {
	ReadLimiter  RateLimiter
	WriteLimiter RateLimiter
}

RateLimiterPair is the actual struct that gets included in every Request type

func (*RateLimiterPair) GetReadRateLimiter added in v1.2.2 

func (rlp *RateLimiterPair) GetReadRateLimiter() RateLimiter

GetReadRateLimiter returns the read limiter for a request note this may be nil

func (*RateLimiterPair) GetWriteRateLimiter added in v1.2.2 

func (rlp *RateLimiterPair) GetWriteRateLimiter() RateLimiter

GetWriteRateLimiter returns the write limiter for a request note this may be nil

func (*RateLimiterPair) SetReadRateLimiter added in v1.2.2 

func (rlp *RateLimiterPair) SetReadRateLimiter(rl RateLimiter)

SetReadRateLimiter sets a read rate limiter instance to use during request execution.

func (*RateLimiterPair) SetWriteRateLimiter added in v1.2.2 

func (rlp *RateLimiterPair) SetWriteRateLimiter(rl RateLimiter)

SetWriteRateLimiter sets a write rate limiter instance to use during request execution.

type RateLimiterPairInt added in v1.2.2 

type RateLimiterPairInt interface {
	GetReadRateLimiter() RateLimiter
	GetWriteRateLimiter() RateLimiter
	SetReadRateLimiter(rl RateLimiter)
	SetWriteRateLimiter(rl RateLimiter)
}

RateLimiterPairInt interface is used in serializer.go to make all Requests include a rate limiter pair

type Region 

type Region string

Region type for OCI regions. 

const (

	// RegionIAD represents the region for US East (Ashburn).
	RegionIAD Region = "us-ashburn-1"
	// RegionPHX represents the region for US West (Phoenix).
	RegionPHX Region = "us-phoenix-1"
	// RegionUSSaltLake2 represents the region for US MidWest (Salt Lake).
	RegionUSSaltLake2 Region = "us-saltlake-2"
	// RegionUSSanJose1 represents the region for US West (San Jose).
	RegionUSSanJose1 Region = "us-sanjose-1"
	// RegionUSChicago1 represents the region for US Central (Chicago).
	RegionUSChicago1 Region = "us-chicago-1"
	// RegionCAMontreal1 represents the region for Canada Southeast (Montreal).
	RegionCAMontreal1 Region = "ca-montreal-1"
	// RegionCAToronto1 represents the region for Canada Southeast (Toronto).
	RegionCAToronto1 Region = "ca-toronto-1"
	// RegionLHR represents the region for UK South (London).
	RegionLHR Region = "uk-london-1"
	// RegionUKCardiff1 represents the region for the UK (Cardiff).
	RegionUKCardiff1 Region = "uk-cardiff-1"
	// RegionEUAmsterdam1 represents the region for Netherlands Northwest (Amsterdam).
	RegionEUAmsterdam1 Region = "eu-amsterdam-1"
	// RegionEUMadrid1 represents the region for Spain (Madrid).
	RegionEUMadrid1 Region = "eu-madrid-1"
	// RegionEUMilan1 represents the region for Italy (Milan).
	RegionEUMilan1 Region = "eu-milan-1"
	// RegionEUParis1 represents the region for France (Paris).
	RegionEUParis1 Region = "eu-paris-1"
	// RegionFRA represents the region for Germany Central (Frankfurt).
	RegionFRA Region = "eu-frankfurt-1"
	// RegionEUZurich1 represents the region for Switzerland North (Zurich).
	RegionEUZurich1 Region = "eu-zurich-1"
	// RegionAPTokyo1 represents the region for Japan East (Tokyo).
	RegionAPTokyo1 Region = "ap-tokyo-1"
	// RegionAPOsaka1 represents the region for Japan Central (Osaka).
	RegionAPOsaka1 Region = "ap-osaka-1"
	// RegionAPSeoul1 represents the region for South Korea Central (Seoul).
	RegionAPSeoul1 Region = "ap-seoul-1"
	// RegionAPChuncheon1 represents the region for South Korea North (Chuncheon).
	RegionAPChuncheon1 Region = "ap-chuncheon-1"
	// RegionAPMumbai1 represents the region for India West (Mumbai).
	RegionAPMumbai1 Region = "ap-mumbai-1"
	// RegionAPHyderabad1 represents the region for India South (Hyderabad).
	RegionAPHyderabad1 Region = "ap-hyderabad-1"
	// RegionAPSydney1 represents the region for Australia East (Sydney).
	RegionAPSydney1 Region = "ap-sydney-1"
	// RegionAPMelbourne1 represents the region for Australia Southeast (Melbourne).
	RegionAPMelbourne1 Region = "ap-melbourne-1"
	// RegionSABogota1 represents the region for Colombia
	RegionSABogota1 Region = "sa-bogota-1"
	// RegionSASaopaulo1 represents the region for Brazil East (Sao Paulo).
	RegionSASaopaulo1 Region = "sa-saopaulo-1"
	// RegionSASantiago1 represents the region for Chile (Santiago).
	RegionSASantiago1 Region = "sa-santiago-1"
	// RegionSAValparaiso1 represents the region for Chile (Valparaiso).
	RegionSAValparaiso1 Region = "sa-valparaiso-1"
	// RegionSAVinhedo1 represents the region for Brazil (Vinhedo).
	RegionSAVinhedo1 Region = "sa-vinhedo-1"
	// RegionMEJeddah1 represents the region for Saudi Arabia West (Jeddah).
	RegionMEJeddah1 Region = "me-jeddah-1"
	// RegionMEDubai1 represents the region for Saudi Arabia East (Dubai).
	RegionMEDubai1 Region = "me-dubai-1"
	// RegionMXQueretaro1 represents the region for Mexico (Queretaro).
	RegionMXQueretaro1 Region = "mx-queretaro-1"
	// RegionMXMonterrey1 represents the region for Mexico (Monterrey).
	RegionMXMonterrey1 Region = "mx-monterrey-1"
	// RegionILJerusalem1 represents the region for Israel (Jerusalem).
	RegionILJerusalem1 Region = "il-jerusalem-1"
	// RegionAFJohannesburg represents the region for Johannesburg
	RegionAFJohannesburg Region = "af-johannesburg-1"
	// RegionAPSignapore represents the region for singapore
	RegionAPSignapore Region = "ap-singapore-1"
	// RegionEUMarseille represents the region for Marseille
	RegionEUMarseille Region = "eu-marseille-1"
	// RegionEUStockholm represents the region for Stockholm
	RegionEUStockholm Region = "eu-stockholm-1"
	// RegionMEAbudhabi represents the region for Abudhabi
	RegionMEAbudhabi Region = "me-abudhabi-1"

	// RegionUSLangley1 represents the region for Langley.
	RegionUSLangley1 Region = "us-langley-1"
	// RegionUSLuke1 represents the region for Luke.
	RegionUSLuke1 Region = "us-luke-1"

	// RegionUSGovAshburn1 represents the government region for Ashburn.
	RegionUSGovAshburn1 Region = "us-gov-ashburn-1"
	// RegionUSGovChicago1 represents the government region for Chicago.
	RegionUSGovChicago1 Region = "us-gov-chicago-1"
	// RegionUSGovPhoenix1 represents the government region for Phoenix.
	RegionUSGovPhoenix1 Region = "us-gov-phoenix-1"

	// RegionUKGovLondon1 represents the government region for London.
	RegionUKGovLondon1 Region = "uk-gov-london-1"
	// RegionUKGovCardiff1 represents the government region for Cardiff.
	RegionUKGovCardiff1 Region = "uk-gov-cardiff-1"

	// RegionUSTacoma1 represents the Tacoma US region
	RegionUSTacoma1 Region = "us-tacoma-1"

	// RegionAPChiyoda1 represents the region for Japan East (Chiyoda).
	RegionAPChiyoda1 Region = "ap-chiyoda-1"
	// RegionAPIbaraki1 represents the region for Japan East (Ibaraki).
	RegionAPIbaraki1 Region = "ap-ibaraki-1"

	// RegionMEDCCMuscat represents the dedicated region for muscat
	RegionMEDCCMuscat Region = "me-dcc-muscat-1"

	// RegionAPDCCCanberra represents the dedicated region for canberra
	RegionAPDCCCanberra Region = "ap-dcc-canberra-1"

	// RegionEUDCCDublin1 represents the dedicated region for Dublin1
	RegionEUDCCDublin1 Region = "eu-dcc-dublin-1"
	// RegionEUDCCDublin2 represents the dedicated region for Dublin2
	RegionEUDCCDublin2 Region = "eu-dcc-dublin-2"
	// RegionEUDCCMilan1 represents the dedicated region for Milan1
	RegionEUDCCMilan1 Region = "eu-dcc-milan-1"
	// RegionEUDCCMilan2 represents the dedicated region for Milan2
	RegionEUDCCMilan2 Region = "eu-dcc-milan-2"
	// RegionEUDCCRating1 represents the dedicated region for Rating1
	RegionEUDCCRating1 Region = "eu-dcc-rating-1"
	// RegionEUDCCRating2 represents the dedicated region for Rating2
	RegionEUDCCRating2 Region = "eu-dcc-rating-2"

	// RegionUSWestJordan1 represents the region for west jordan (Utah)
	RegionUSWestJordan1 Region = "us-westjordan-1"

	// RegionUSDCCPhoenix1 represents dedicated region 1 for Phoenix
	RegionDCCPhoenix1 Region = "us-dcc-phoenix-1"
	// RegionUSDCCPhoenix2 represents dedicated region 2 for Phoenix
	RegionDCCPhoenix2 Region = "us-dcc-phoenix-2"
	// RegionUSDCCPhoenix4 represents dedicated region 4 for Phoenix
	RegionDCCPhoenix4 Region = "us-dcc-phoenix-4"

	// RegionEUFrankfurt2 represents the region for Frankfurt 2
	RegionEUFrankfurt2 Region = "eu-frankfurt-2"
	// RegionEUMadrid2 represents the region for Madrid 2
	RegionEUMadrid2 Region = "eu-madrid-2"

	// RegionEUJovanovac1 represents the region for Jovanovac 1 (Serbia)
	RegionEUJovanovac1 Region = "eu-jovanovac-1"

	// RegionEUJovanovac1 represents the dedicated region for Rome
	RegionEUDCCRome1 Region = "eu-dcc-rome-1"

	// RegionEUDCCZurich1 represents the dedicated region for Zurich
	RegionEUDCCZurich1 Region = "eu-dcc-zurich-1"

	// RegionUSDCCSWJordan1 represents the dedicated region for SWJordan1
	RegionUSDCCSWJordan1 Region = "us-dcc-swjordan-1"

	// RegionUSDCCSWJordan1 represents the dedicated region for SWJordan2
	RegionUSDCCSWJordan2 Region = "us-dcc-swjordan-2"
)

func StringToRegion 

func StringToRegion(regionKeyOrID string) (r Region, err error)

StringToRegion converts a string that represents the region key or region identifier to Region type. The supported region keys and region identifiers are available at https://docs.cloud.oracle.com/iaas/Content/General/Concepts/regions.htm.

func (Region) Endpoint 

func (region Region) Endpoint() (string, error)

Endpoint returns the NoSQL service endpoint for the region. An error is returned if region is invalid.

func (Region) EndpointForService 

func (region Region) EndpointForService(service string) (string, error)

EndpointForService returns an endpoint for a service. An error is returned if region is invalid.

type SimpleRateLimiter added in v1.2.2 

type SimpleRateLimiter struct {
	// contains filtered or unexported fields
}

SimpleRateLimiter is an implementation of RateLimiter interface. Despite its name, it also has methods for percentage-based operations as well as the methods in RateLimiter. Currently the percentage-based methods are only used internally.

func NewSimpleRateLimiter added in v1.2.2 

func NewSimpleRateLimiter(rateLimitPerSec float64) (srl *SimpleRateLimiter)

NewSimpleRateLimiter Creates a simple time-based rate limiter.

This limiter will allow for one second of duration; that is, it will allow unused units from within the last second to be used.

func NewSimpleRateLimiterWithDuration added in v1.2.2 

func NewSimpleRateLimiterWithDuration(rateLimitPerSec float64, durationSecs float64) (srl *SimpleRateLimiter)

NewSimpleRateLimiterWithDuration creates a simple time-based rate limiter with a specified duration.

func (*SimpleRateLimiter) ConsumeUnits added in v1.2.2 

func (srl *SimpleRateLimiter) ConsumeUnits(units int64) time.Duration

ConsumeUnits consumes a number of units, blocking until the units are available. It returns the amount of time blocked. If not blocked, 0 is returned.

func (*SimpleRateLimiter) ConsumeUnitsUnconditionally added in v1.2.2 

func (srl *SimpleRateLimiter) ConsumeUnitsUnconditionally(units int64)

ConsumeUnitsUnconditionally consumes units without checking or waiting. The internal amount of units consumed will be updated by the given amount, regardless of its current over/under limit state. The number of units to consume may be negative to "give back" units.

func (*SimpleRateLimiter) ConsumeUnitsWithTimeout added in v1.2.2 

func (srl *SimpleRateLimiter) ConsumeUnitsWithTimeout(units int64, timeout time.Duration, alwaysConsume bool) (time.Duration, error)

ConsumeUnitsWithTimeout attempts to consume a number of units, blocking until the units are available or the specified timeout expires. It returns the amount of time blocked. If not blocked, 0 is returned.

units can be a negative value to "give back" units. Pass a timeout value of 0 to block indefinitely. To poll if the limiter is currently over its limit, use TryConsumeUnits() instead. If alwaysConsume is true, consume units even on timeout.

func (*SimpleRateLimiter) GetCurrentRate added in v1.2.2 

func (srl *SimpleRateLimiter) GetCurrentRate() float64

GetCurrentRate returns the current rate as a percentage of current limit (0.0 - 100.0).

func (*SimpleRateLimiter) GetDuration added in v1.2.2 

func (srl *SimpleRateLimiter) GetDuration() float64

GetDuration returns the duration in seconds configured for this rate limiter instance

func (*SimpleRateLimiter) GetLimitPerSecond added in v1.2.2 

func (srl *SimpleRateLimiter) GetLimitPerSecond() float64

GetLimitPerSecond returns the number of units configured for this rate limiter instance (the max number of units per second this limiter allows).

func (*SimpleRateLimiter) Reset added in v1.2.2 

func (srl *SimpleRateLimiter) Reset()

Reset resets the rate limiter as if it was newly constructed.

func (*SimpleRateLimiter) SetCurrentRate added in v1.2.2 

func (srl *SimpleRateLimiter) SetCurrentRate(percent float64)

SetCurrentRate sets the current rate as a percentage of current limit. This modifies the internal limiter values; it does not modify the rate limit. rateToSet may be greater than 100.0 to set the limiter to "over its limit".

func (*SimpleRateLimiter) SetDuration added in v1.2.2 

func (srl *SimpleRateLimiter) SetDuration(durationSecs float64)

SetDuration sets the duration for this rate limiter instance. The duration specifies how far back in time the limiter will go to consume previously unused units.

For example, if a limiter had a limit of 1000 units and a duration of 5 seconds, and had no units consumed for at least 5 seconds, a call to TryConsumeUnits(5000) will succeed immediately with no waiting.

func (*SimpleRateLimiter) SetLimitPerSecond added in v1.2.2 

func (srl *SimpleRateLimiter) SetLimitPerSecond(rateLimitPerSec float64)

SetLimitPerSecond sets a new limit (units per second) on the limiter. Changing the limit may lead to unexpected spiky behavior, and may affect other goroutines currently operating on the same limiter instance.

func (*SimpleRateLimiter) String added in v1.2.2 

func (srl *SimpleRateLimiter) String() string

func (*SimpleRateLimiter) TryConsumeUnits added in v1.2.2 

func (srl *SimpleRateLimiter) TryConsumeUnits(units int64) bool

TryConsumeUnits consumes the specified number of units if they can be returned immediately without waiting. It returns true if the units were consumed, false if they were not immediately available. If units was zero, true return means the limiter is currently below its limit. units can be zero to poll if the limiter is currently over its limit. Pass negative units to "give back" units (same as calling ConsumeUnits with a negative value).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.