util

package
v2.0.0-...-439aceb Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 15, 2023 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// InternalTxnOthers is the type of requests that consume low resources.
	// This reduces the size of metrics.
	InternalTxnOthers = "others"
	// InternalTxnGC is the type of GC txn.
	InternalTxnGC = "gc"
	// InternalTxnMeta is the type of the miscellaneous meta usage.
	InternalTxnMeta = InternalTxnOthers
)
View Source
const (
	// InternalRequest is the scope of internal queries
	InternalRequest = "internal_"
	// ExternalRequest is the scope of external queries
	ExternalRequest = "external_"
	// SourceUnknown keeps same with the default value(empty string)
	SourceUnknown = "unknown"
)
View Source
const GCTimeFormat = "20060102-15:04:05.000 -0700"

GCTimeFormat is the format that gc_worker used to store times.

Variables

View Source
var (
	// CommitDetailCtxKey presents CommitDetail info key in context.
	CommitDetailCtxKey = commitDetailCtxKeyType{}

	// LockKeysDetailCtxKey presents LockKeysDetail info key in context.
	LockKeysDetailCtxKey = lockKeysDetailCtxKeyType{}

	// ExecDetailsKey presents ExecDetail info key in context.
	ExecDetailsKey = execDetailsCtxKeyType{}
)
View Source
var RequestSourceKey = RequestSourceKeyType{}

RequestSourceKey is used as the key of request source type in context.

View Source
var RequestSourceTypeKey = RequestSourceTypeKeyType{}

RequestSourceTypeKey is used as the key of request source type in context.

View Source
var SessionID = sessionIDCtxKey{}

SessionID is the context key type to mark a session.

Functions

func BytesToString

func BytesToString(numBytes int64) string

BytesToString converts the memory consumption to a readable string.

func CompatibleParseGCTime

func CompatibleParseGCTime(value string) (time.Time, error)

CompatibleParseGCTime parses a string with `GCTimeFormat` and returns a time.Time. If `value` can't be parsed as that format, truncate to last space and try again. This function is only useful when loading times that saved by gc_worker. We have changed the format that gc_worker saves time (removed the last field), but when loading times it should be compatible with the old format.

func ContextWithTraceExecDetails

func ContextWithTraceExecDetails(ctx context.Context) context.Context

ContextWithTraceExecDetails returns a context with trace-exec-details enabled

func EnableFailpoints

func EnableFailpoints()

EnableFailpoints enables use of failpoints. It should be called before using client to avoid data race.

func EncodeToString

func EncodeToString(src []byte) []byte

EncodeToString overrides hex.EncodeToString implementation. Difference: returns []byte, not string

func EvalFailpoint

func EvalFailpoint(name string) (interface{}, error)

EvalFailpoint injects code for testing. It is used to replace `failpoint.Inject` to make it possible to be used in a library.

func FormatBytes

func FormatBytes(numBytes int64) string

FormatBytes uses to format bytes, this function will prune precision before format bytes.

func FormatDuration

func FormatDuration(d time.Duration) string

FormatDuration uses to format duration, this function will prune precision before format duration. Pruning precision is for human readability. The prune rule is:

  1. if the duration was less than 1us, return the original string.
  2. readable value >=10, keep 1 decimal, otherwise, keep 2 decimal. such as: 9.412345ms -> 9.41ms 10.412345ms -> 10.4ms 5.999s -> 6s 100.45µs -> 100.5µs

func HexRegionKey

func HexRegionKey(key []byte) []byte

HexRegionKey converts region key to hex format. Used for formating region in logs.

func HexRegionKeyStr

func HexRegionKeyStr(key []byte) string

HexRegionKeyStr converts region key to hex format. Used for formating region in logs.

func RequestSourceFromCtx

func RequestSourceFromCtx(ctx context.Context) string

RequestSourceFromCtx extract source from passed context.

func ResourceGroupNameFromCtx

func ResourceGroupNameFromCtx(ctx context.Context) string

ResourceGroupNameFromCtx extract resource group name from passed context, empty string is returned is the key is not set.

func SetSessionID

func SetSessionID(ctx context.Context, sessionID uint64) context.Context

SetSessionID sets session id into context

func String

func String(b []byte) (s string)

String converts slice of bytes to string without copy.

func ToUpperASCIIInplace

func ToUpperASCIIInplace(s []byte) []byte

ToUpperASCIIInplace bytes.ToUpper but zero-cost

func TraceExecDetailsEnabled

func TraceExecDetailsEnabled(ctx context.Context) bool

TraceExecDetailsEnabled checks whether trace-exec-details enabled

func WithInternalSourceType

func WithInternalSourceType(ctx context.Context, source string) context.Context

WithInternalSourceType create context with internal source.

func WithRecovery

func WithRecovery(exec func(), recoverFn func(r interface{}))

WithRecovery wraps goroutine startup call with force recovery. it will dump current goroutine stack into log if catch any recover result.

exec:      execute logic function.
recoverFn: handler will be called after recover and before dump stack, passing `nil` means noop.

func WithResouceGroupName

func WithResouceGroupName(ctx context.Context, groupName string) context.Context

WithResouceGroupName return a copy of the given context with a associated reosurce group name.

Types

type CommitDetails

type CommitDetails struct {
	GetCommitTsTime        time.Duration
	GetLatestTsTime        time.Duration
	PrewriteTime           time.Duration
	WaitPrewriteBinlogTime time.Duration
	CommitTime             time.Duration
	LocalLatchTime         time.Duration
	Mu                     struct {
		sync.Mutex
		// The total backoff time used in both the prewrite and commit phases.
		CommitBackoffTime    int64
		PrewriteBackoffTypes []string
		CommitBackoffTypes   []string
		// The prewrite requests are executed concurrently so the slowest request information would be recorded.
		SlowestPrewrite ReqDetailInfo
		// It's recorded only when the commit mode is 2pc.
		CommitPrimary ReqDetailInfo
	}
	WriteKeys         int
	WriteSize         int
	PrewriteRegionNum int32
	TxnRetry          int
	ResolveLock       ResolveLockDetail
	PrewriteReqNum    int
}

CommitDetails contains commit detail information.

func (*CommitDetails) Clone

func (cd *CommitDetails) Clone() *CommitDetails

Clone returns a deep copy of itself.

func (*CommitDetails) Merge

func (cd *CommitDetails) Merge(other *CommitDetails)

Merge merges commit details into itself.

func (*CommitDetails) MergeCommitReqDetails

func (cd *CommitDetails) MergeCommitReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergeCommitReqDetails merges commit related ExecDetailsV2 into the current CommitDetails.

func (*CommitDetails) MergePrewriteReqDetails

func (cd *CommitDetails) MergePrewriteReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergePrewriteReqDetails merges prewrite related ExecDetailsV2 into the current CommitDetails.

type ExecDetails

type ExecDetails struct {
	BackoffCount       int64
	BackoffDuration    int64
	WaitKVRespDuration int64
	WaitPDRespDuration int64
}

ExecDetails contains execution detail info.

type InterceptedPDClient

type InterceptedPDClient struct {
	pd.Client
}

InterceptedPDClient is a PD's wrapper client to record stmt detail.

func (InterceptedPDClient) GetPrevRegion

func (m InterceptedPDClient) GetPrevRegion(ctx context.Context, key []byte, opts ...pd.GetRegionOption) (*pd.Region, error)

GetPrevRegion implements pd.Client#GetPrevRegion.

func (InterceptedPDClient) GetRegion

func (m InterceptedPDClient) GetRegion(ctx context.Context, key []byte, opts ...pd.GetRegionOption) (*pd.Region, error)

GetRegion implements pd.Client#GetRegion.

func (InterceptedPDClient) GetRegionByID

func (m InterceptedPDClient) GetRegionByID(ctx context.Context, regionID uint64, opts ...pd.GetRegionOption) (*pd.Region, error)

GetRegionByID implements pd.Client#GetRegionByID.

func (InterceptedPDClient) GetStore

func (m InterceptedPDClient) GetStore(ctx context.Context, storeID uint64) (*metapb.Store, error)

GetStore implements pd.Client#GetStore.

func (InterceptedPDClient) GetTS

GetTS implements pd.Client#GetTS.

func (InterceptedPDClient) GetTSAsync

func (m InterceptedPDClient) GetTSAsync(ctx context.Context) pd.TSFuture

GetTSAsync implements pd.Client#GetTSAsync.

func (InterceptedPDClient) ScanRegions

func (m InterceptedPDClient) ScanRegions(ctx context.Context, key, endKey []byte, limit int) ([]*pd.Region, error)

ScanRegions implements pd.Client#ScanRegions.

type LockKeysDetails

type LockKeysDetails struct {
	TotalTime   time.Duration
	RegionNum   int32
	LockKeys    int32
	ResolveLock ResolveLockDetail
	BackoffTime int64
	Mu          struct {
		sync.Mutex
		BackoffTypes        []string
		SlowestReqTotalTime time.Duration
		SlowestRegion       uint64
		SlowestStoreAddr    string
		SlowestExecDetails  TiKVExecDetails
	}
	LockRPCTime  int64
	LockRPCCount int64
	RetryCount   int
}

LockKeysDetails contains pessimistic lock keys detail information.

func (*LockKeysDetails) Clone

func (ld *LockKeysDetails) Clone() *LockKeysDetails

Clone returns a deep copy of itself.

func (*LockKeysDetails) Merge

func (ld *LockKeysDetails) Merge(lockKey *LockKeysDetails)

Merge merges lock keys execution details into self.

func (*LockKeysDetails) MergeReqDetails

func (ld *LockKeysDetails) MergeReqDetails(reqDuration time.Duration, regionID uint64, addr string, execDetails *kvrpcpb.ExecDetailsV2)

MergeReqDetails merges ExecDetailsV2 into the current LockKeysDetails.

type RateLimit

type RateLimit struct {
	// contains filtered or unexported fields
}

RateLimit wraps a fix sized channel to control concurrency.

func NewRateLimit

func NewRateLimit(n int) *RateLimit

NewRateLimit creates a limit controller with capacity n.

func (*RateLimit) GetCapacity

func (r *RateLimit) GetCapacity() int

GetCapacity returns the token capacity.

func (*RateLimit) GetToken

func (r *RateLimit) GetToken(done <-chan struct{}) (exit bool)

GetToken acquires a token.

func (*RateLimit) PutToken

func (r *RateLimit) PutToken()

PutToken puts a token back.

type ReqDetailInfo

type ReqDetailInfo struct {
	ReqTotalTime time.Duration
	Region       uint64
	StoreAddr    string
	ExecDetails  TiKVExecDetails
}

ReqDetailInfo contains diagnose information about `TiKVExecDetails`, region, store and backoff.

type RequestSource

type RequestSource struct {
	RequestSourceInternal bool
	RequestSourceType     string
}

RequestSource contains the source label of the request, used for tracking resource consuming.

func (*RequestSource) GetRequestSource

func (r *RequestSource) GetRequestSource() string

GetRequestSource gets the request_source field of the request.

func (*RequestSource) SetRequestSourceInternal

func (r *RequestSource) SetRequestSourceInternal(internal bool)

SetRequestSourceInternal sets the scope of the request source.

func (*RequestSource) SetRequestSourceType

func (r *RequestSource) SetRequestSourceType(tp string)

SetRequestSourceType sets the type of the request source.

type RequestSourceKeyType

type RequestSourceKeyType struct{}

RequestSourceKeyType is a dummy type to avoid naming collision in context.

type RequestSourceTypeKeyType

type RequestSourceTypeKeyType struct{}

RequestSourceTypeKeyType is a dummy type to avoid naming collision in context.

type ResolveLockDetail

type ResolveLockDetail struct {
	// ResolveLockTime is the total duration of resolving lock.
	ResolveLockTime int64
}

ResolveLockDetail contains the resolve lock detail information.

func (*ResolveLockDetail) Merge

func (rd *ResolveLockDetail) Merge(resolveLock *ResolveLockDetail)

Merge merges resolve lock detail details into self.

type ScanDetail

type ScanDetail struct {
	// TotalKeys is the approximate number of MVCC keys meet during scanning. It includes
	// deleted versions, but does not include RocksDB tombstone keys.
	TotalKeys int64
	// ProcessedKeys is the number of user keys scanned from the storage.
	// It does not include deleted version or RocksDB tombstone keys.
	// For Coprocessor requests, it includes keys that has been filtered out by Selection.
	ProcessedKeys int64
	// Number of bytes of user key-value pairs scanned from the storage, i.e.
	// total size of data returned from MVCC layer.
	ProcessedKeysSize int64
	// RocksdbDeleteSkippedCount is the total number of deletes and single deletes skipped over during
	// iteration, i.e. how many RocksDB tombstones are skipped.
	RocksdbDeleteSkippedCount uint64
	// RocksdbKeySkippedCount it the total number of internal keys skipped over during iteration.
	RocksdbKeySkippedCount uint64
	// RocksdbBlockCacheHitCount is the total number of RocksDB block cache hits.
	RocksdbBlockCacheHitCount uint64
	// RocksdbBlockReadCount is the total number of block reads (with IO).
	RocksdbBlockReadCount uint64
	// RocksdbBlockReadByte is the total number of bytes from block reads.
	RocksdbBlockReadByte uint64
	// RocksdbBlockReadDuration is the total time used for block reads.
	RocksdbBlockReadDuration time.Duration
	// GetSnapshotDuration is the time spent getting an engine snapshot.
	GetSnapshotDuration time.Duration

	ResolveLock *ResolveLockDetail
}

ScanDetail contains coprocessor scan detail information.

func (*ScanDetail) Merge

func (sd *ScanDetail) Merge(scanDetail *ScanDetail)

Merge merges scan detail execution details into self.

func (*ScanDetail) MergeFromScanDetailV2

func (sd *ScanDetail) MergeFromScanDetailV2(scanDetail *kvrpcpb.ScanDetailV2)

MergeFromScanDetailV2 merges scan detail from pb into itself.

func (*ScanDetail) String

func (sd *ScanDetail) String() string

String implements the fmt.Stringer interface.

type TSSet

type TSSet struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

TSSet is a set of timestamps.

func (*TSSet) GetAll

func (s *TSSet) GetAll() []uint64

GetAll returns all timestamps in the set.

func (*TSSet) Put

func (s *TSSet) Put(tss ...uint64)

Put puts timestamps into the map.

type TiKVExecDetails

type TiKVExecDetails struct {
	TimeDetail  *TimeDetail
	ScanDetail  *ScanDetail
	WriteDetail *WriteDetail
}

TiKVExecDetails is the detail execution information at TiKV side.

func NewTiKVExecDetails

func NewTiKVExecDetails(pb *kvrpcpb.ExecDetailsV2) TiKVExecDetails

NewTiKVExecDetails creates a TiKVExecDetails from a kvproto ExecDetailsV2.

func (*TiKVExecDetails) String

func (ed *TiKVExecDetails) String() string

type TimeDetail

type TimeDetail struct {
	// Off-cpu and on-cpu wall time elapsed to actually process the request payload. It does not
	// include `wait_wall_time`.
	// This field is very close to the CPU time in most cases. Some wait time spend in RocksDB
	// cannot be excluded for now, like Mutex wait time, which is included in this field, so that
	// this field is called wall time instead of CPU time.
	ProcessTime time.Duration
	// Off-cpu wall time elapsed in TiKV side. Usually this includes queue waiting time and
	// other kind of waits in series.
	WaitTime time.Duration
	// KvReadWallTimeMs is the time used in KV Scan/Get.
	KvReadWallTimeMs time.Duration
	// TotalRPCWallTime is Total wall clock time spent on this RPC in TiKV.
	TotalRPCWallTime time.Duration
}

TimeDetail contains coprocessor time detail information.

func (*TimeDetail) MergeFromTimeDetail

func (td *TimeDetail) MergeFromTimeDetail(timeDetail *kvrpcpb.TimeDetail)

MergeFromTimeDetail merges time detail from pb into itself.

func (*TimeDetail) String

func (td *TimeDetail) String() string

String implements the fmt.Stringer interface.

type WriteDetail

type WriteDetail struct {
	// StoreBatchWaitDuration is the wait duration in the store loop.
	StoreBatchWaitDuration time.Duration
	// ProposeSendWaitDuration is the duration before sending proposal to peers.
	ProposeSendWaitDuration time.Duration
	// PersistLogDuration is the total time spent on persisting the log.
	PersistLogDuration time.Duration
	// RaftDbWriteLeaderWaitDuration is the wait time until the Raft log write leader begins to write.
	RaftDbWriteLeaderWaitDuration time.Duration
	// RaftDbSyncLogDuration is the time spent on synchronizing the Raft log to the disk.
	RaftDbSyncLogDuration time.Duration
	// RaftDbWriteMemtableDuration is the time spent on writing the Raft log to the Raft memtable.
	RaftDbWriteMemtableDuration time.Duration
	// CommitLogDuration is the time waiting for peers to confirm the proposal (counting from the instant when the leader sends the proposal message).
	CommitLogDuration time.Duration
	// ApplyBatchWaitDuration is the wait duration in the apply loop.
	ApplyBatchWaitDuration time.Duration
	// ApplyLogDuration is the total time spend to applying the log.
	ApplyLogDuration time.Duration
	// ApplyMutexLockDuration is the wait time until the KV RocksDB lock is acquired.
	ApplyMutexLockDuration time.Duration
	// ApplyWriteLeaderWaitDuration is the wait time until becoming the KV RocksDB write leader.
	ApplyWriteLeaderWaitDuration time.Duration
	// ApplyWriteWalDuration is the time spent on writing the KV DB WAL to the disk.
	ApplyWriteWalDuration time.Duration
	// ApplyWriteMemtableNanos is the time spent on writing to the memtable of the KV RocksDB.
	ApplyWriteMemtableDuration time.Duration
}

WriteDetail contains the detailed time breakdown of a write operation.

func (*WriteDetail) Merge

func (wd *WriteDetail) Merge(writeDetail *WriteDetail)

Merge merges another WriteDetail protobuf into self.

func (*WriteDetail) MergeFromWriteDetailPb

func (wd *WriteDetail) MergeFromWriteDetailPb(pb *kvrpcpb.WriteDetail)

MergeFromWriteDetailPb merges WriteDetail protobuf into the current WriteDetail

func (*WriteDetail) String

func (wd *WriteDetail) String() string

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL