Documentation ¶
Index ¶
- Constants
- Variables
- func GetElem(doc bson.D, key string) (interface{}, bool)
- func NewBestEffortOpsDispatcher(reader OpsReader, opsSize int, logger *Logger) chan *Op
- func NewByTimeOpsDispatcher(reader OpsReader, opsSize int, logger *Logger, speedup float64) chan *Op
- type ByLineOpsReader
- func (r *ByLineOpsReader) AllLoaded() bool
- func (r *ByLineOpsReader) Close()
- func (r *ByLineOpsReader) Err() error
- func (r *ByLineOpsReader) Next() *Op
- func (r *ByLineOpsReader) OpsRead() int
- func (r *ByLineOpsReader) SetStartTime(startTime int64) (int64, error)
- func (r *ByLineOpsReader) SkipOps(numSkipOps int) error
- type CyclicOpsReader
- func (c *CyclicOpsReader) AllLoaded() bool
- func (c *CyclicOpsReader) Close()
- func (c *CyclicOpsReader) Err() error
- func (c *CyclicOpsReader) Next() *Op
- func (c *CyclicOpsReader) OpsRead() int
- func (c *CyclicOpsReader) SetStartTime(startTime int64) (int64, error)
- func (c *CyclicOpsReader) SkipOps(numSkipOps int) error
- type Document
- type ExecutionStatus
- type Logger
- type Op
- type OpStat
- type OpType
- type OpsExecutor
- type OpsReader
- type StatsAnalyzer
Constants ¶
Percentiles
Variables ¶
var AllOpTypes = []OpType{ Insert, Update, Remove, Query, Count, FindAndModify, GetMore, }
AllOpTypes specifies all supported op types
var (
NotSupported = errors.New("op type not supported")
)
Functions ¶
func GetElem ¶
GetElem is a helper to fetch a specific key from bson.D The second return value indicates whether or not the key exists
Types ¶
type ByLineOpsReader ¶
type ByLineOpsReader struct {
// contains filtered or unexported fields
}
ByLineOpsReader reads ops from a json file that is exported from python's json_util module, where each line is a json-represented op.
Note: After parse each json-represented op, we need perform post-process to convert some "metadata" into MongoDB specific data structures, like "Object Id" and datetime.
func NewByLineOpsReader ¶
func NewByLineOpsReader(reader io.ReadCloser, logger *Logger, opFilter string) (error, *ByLineOpsReader)
func NewFileByLineOpsReader ¶
func NewFileByLineOpsReader(filename string, logger *Logger, opFilter string) (error, *ByLineOpsReader)
func (*ByLineOpsReader) AllLoaded ¶
func (r *ByLineOpsReader) AllLoaded() bool
func (*ByLineOpsReader) Close ¶
func (r *ByLineOpsReader) Close()
func (*ByLineOpsReader) Err ¶
func (r *ByLineOpsReader) Err() error
func (*ByLineOpsReader) Next ¶
func (r *ByLineOpsReader) Next() *Op
func (*ByLineOpsReader) OpsRead ¶
func (r *ByLineOpsReader) OpsRead() int
func (*ByLineOpsReader) SetStartTime ¶
func (r *ByLineOpsReader) SetStartTime(startTime int64) (int64, error)
func (*ByLineOpsReader) SkipOps ¶
func (r *ByLineOpsReader) SkipOps(numSkipOps int) error
type CyclicOpsReader ¶
type CyclicOpsReader struct {
// contains filtered or unexported fields
}
func NewCyclicOpsReader ¶
func NewCyclicOpsReader(maker func() OpsReader, logger *Logger) *CyclicOpsReader
func (*CyclicOpsReader) AllLoaded ¶
func (c *CyclicOpsReader) AllLoaded() bool
func (*CyclicOpsReader) Close ¶
func (c *CyclicOpsReader) Close()
func (*CyclicOpsReader) Err ¶
func (c *CyclicOpsReader) Err() error
func (*CyclicOpsReader) Next ¶
func (c *CyclicOpsReader) Next() *Op
func (*CyclicOpsReader) OpsRead ¶
func (c *CyclicOpsReader) OpsRead() int
func (*CyclicOpsReader) SetStartTime ¶
func (c *CyclicOpsReader) SetStartTime(startTime int64) (int64, error)
func (*CyclicOpsReader) SkipOps ¶
func (c *CyclicOpsReader) SkipOps(numSkipOps int) error
type Document ¶
type Document map[string]interface{}
Document represents the json-like infromation of an op
type ExecutionStatus ¶
type ExecutionStatus struct { OpsExecuted int64 IntervalOpsExecuted int64 OpsErrors int64 IntervalOpsErrors int64 OpsPerSec float64 IntervalOpsPerSec float64 IntervalDuration time.Duration Latencies map[OpType][]float64 IntervalLatencies map[OpType][]float64 MaxLatency map[OpType]float64 IntervalMaxLatency map[OpType]float64 Counts map[OpType]int64 IntervalCounts map[OpType]int64 TypeOpsSec map[OpType]float64 IntervalTypeOpsSec map[OpType]float64 }
ExecutionStatus encapsulates the aggregated information for the execution
type Logger ¶
type Logger struct {
// contains filtered or unexported fields
}
Logger provides a way to send different types of log messages to stderr/stdout
type Op ¶
type Op struct { Ns string `bson:"ns"` Timestamp time.Time `bson:"ts"` Type OpType `bson:"op"` NToSkip int64 `bson:"ntoskip,omitempty"` NToReturn int64 `bson:"ntoreturn,omitempty"` QueryDoc bson.D `bson:"query,omitempty"` CommandDoc bson.D `bson:"command,omitempty"` InsertDoc bson.D `bson:"o,omitempty"` UpdateDoc bson.D `bson:"updateobj,omitempty"` Database string `bson:",omitempty"` Collection string `bson:",omitempty"` }
Op represents an op generated by the record utility It must (currently) be massaged a little before handing off to the executor
func CanonicalizeOp ¶
We only support handful op types. This function helps us to process supported ops in a universal way.
We do not canonicalize the ops in OpsReader because we hope ops reader to do its job honestly and the consumer of these ops decide how to further process the original ops.
type OpsExecutor ¶
type OpsExecutor struct {
// contains filtered or unexported fields
}
func NewOpsExecutor ¶
func NewOpsExecutor(session *mgo.Session, statsChan chan OpStat, logger *Logger) *OpsExecutor
func (*OpsExecutor) Execute ¶
func (e *OpsExecutor) Execute(op *Op) error
func (*OpsExecutor) LastLatency ¶
func (e *OpsExecutor) LastLatency() time.Duration
type OpsReader ¶
type OpsReader interface { // Move to next op and return it. Nil will be returned if the last ops had // already been read, or there is any error occurred. // TODO change from Document to Op Next() *Op // Allow skipping the first N ops in the source file SkipOps(int) error // Start at a specific time in the set of ops // Return an error if we get to EOF without finding an op // Can be used with SkipOps, but you should call SkipOps after SetStartTime SetStartTime(int64) (int64, error) // How many ops are read so far OpsRead() int // Have all the ops been read? AllLoaded() bool // indicate the latest error occurs when reading ops. Err() error Close() }
OpsReader Reads the ops from a source and present a interface for consumers to fetch these ops sequentially.
type StatsAnalyzer ¶
type StatsAnalyzer struct {
// contains filtered or unexported fields
}
func NewStatsAnalyzer ¶
func NewStatsAnalyzer(statsChan chan OpStat) *StatsAnalyzer
func (*StatsAnalyzer) GetStatus ¶
func (s *StatsAnalyzer) GetStatus() *ExecutionStatus
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
pcap_converter
Simple program which accepts a pcap file and prints a flashback-compatible ops stream
|
Simple program which accepts a pcap file and prints a flashback-compatible ops stream |