Documentation
¶
Overview ¶
Package endpoint contains stats for client endpoints. An endpoint corresponds to an mlab-ns request signature that we expect may represent an individual requester. (IP alone is insufficient, because of CG-NAT and use of proxies). We use the userAgent, resource string, and IP address. NOTE: We currently limit results to 20K endpoints, and as of Sept 2018, we are seeing about 8K endpoints with more than 12 requests per day. The limit is imposed because we don't have enough experience to predict how mlab-ns might behave if bad endpoints grew to 100K or 200K.
Index ¶
- func CalculateClientSignature(ip string, userAgent string, resource string) string
- func DSKey(name string) *datastore.Key
- func DeleteAllKeys(ctx context.Context, client *datastore.Client, namespace string, kind string) (int, error)
- func FetchEndpointStats(ctx context.Context, dsExt *bqx.Dataset, threshold int) ([]map[string]bigquery.Value, error)
- func GetAllKeys(ctx context.Context, client *datastore.Client, namespace string, kind string) ([]*datastore.Key, error)
- func PutMulti(ctx context.Context, client *datastore.Client, keys []*datastore.Key, ...) error
- func SetMulti(c context.Context, keys []*datastore.Key, endpoints []Stats) error
- type Stats
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CalculateClientSignature ¶ added in v0.2.1
CalculateClientSignature generates a signature based on the client IP, UserAgent and requested resource. The resulting signature must not be longer than 250 bytes so it can be used as a memcache key (having MAX_LENGTH=250).
func DeleteAllKeys ¶
func DeleteAllKeys(ctx context.Context, client *datastore.Client, namespace string, kind string) (int, error)
DeleteAllKeys deletes all keys for a namespace and kind from Datastore, dividing into blocks of 500 to satisfy datastore API constraint. If there are errors, it returns the last error message, but the length returned will not reflect the failures.
func FetchEndpointStats ¶
func FetchEndpointStats(ctx context.Context, dsExt *bqx.Dataset, threshold int) ([]map[string]bigquery.Value, error)
FetchEndpointStats executes simpleQuery, and returns a slice of rows containing endpoint signatures and request counts. TODO - move the body (excluding simpleQuery) into go/bqext
func GetAllKeys ¶
func GetAllKeys(ctx context.Context, client *datastore.Client, namespace string, kind string) ([]*datastore.Key, error)
GetAllKeys fetches all keys from Datastore for a namespace and kind.
func PutMulti ¶
func PutMulti(ctx context.Context, client *datastore.Client, keys []*datastore.Key, endpoints []Stats) error
PutMulti writes a set of keys and endpoints to datastore, dividing into blocks of 500 to satisfy datastore API constraint.
func SetMulti ¶ added in v0.2.0
SetMulti writes the keys and endpoints Probability. To preserve cross-language compatibility, SetMulti formats the endpoints Probability (float) as a string encoded integer. Readers should parse the value as an int, then divide by 10000 to recover the original probability.
Types ¶
type Stats ¶
type Stats struct {
RequestsPerDay int64 `datastore:"requests_per_day"` // Number of requests made per day.
Probability float32 `datastore:"probability"` // Fraction of requests that should be sent to standard backend.
}
Stats contains information about request rate for an endpoint, and probability for mlab-ns to use in routing new requests.
func MakeKeysAndStats ¶
func MakeKeysAndStats(rows []map[string]bigquery.Value, threshold int) ([]*datastore.Key, []Stats, error)
MakeKeysAndStats converts slice of bigquery rows into DSKeys and Stats objects.
func StatsFromMap ¶
StatsFromMap creates a Key and Stats object from a bigquery result map.
Source Files