ftdc

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 16, 2023 License: Apache-2.0 Imports: 24 Imported by: 0

README

MongoDB FTDC Metrics and Charts

A dockerized tool to view MongoDB FTDC metrics.

Build

Use build.sh to build simagix/ftdc and simagix/grafana-ftdc Docker images.

./build.sh docker

Startup

Create a diagnostic.data directory if it doesn't exist yet:

mkdir -p ./diagnostic.data/

Copy FTDC files to under directory diagnostic.data:

cp $SOMEWHERE/metrics.* ./diagnostic.data/

Bring up FTDC viewer:

docker-compose up -d

View FTDC Metrics

  • View results URL http://localhost:3030/ using a browser.
  • Choose MongoDB FTDC Analytics from dashboard.
  • Change correct From and To date/time in the Custom Range panel.

Read Other FTDC Data

To read different FTDC files without restarting all Docker containers, remove all files from directory diagnostic.data and copy FTDC files to under the same directory. Execute the command below to force FTDC data reload:

curl -XPOST http://localhost:5408/grafana/dir -d '{"dir": "/diagnostic.data"}'

A JSON document is returned with information of the new endpoint, begin and end timestamps:

{"endpoint":"/d/simagix-grafana/mongodb-mongo-ftdc?orgId=1\u0026from=1550767345000\u0026to=1550804249000","ok":1}

Alternatively, you can simply run the command docker-compose down to bring down the entire cluster, edit the docker-compose.yaml file to point the diagnostic.data directory to a new location, and run docker-compose up to bring up the cluster again.

Shutdown

docker-compose down

Disclaimer

This software is not supported by MongoDB, Inc. under any of their commercial support subscriptions or otherwise. Any usage is at your own risk. Bug reports, feature requests and questions can be posted in the Issues section on GitHub.

Documentation

Index

Constants

View Source 
const PRIMARY = "PRIMARY"

PRIMARY - primary node

View Source 
const SECONDARY = "SECONDARY"

SECONDARY - secondary node

Variables

View Source 
var FormulaMap = map[string]ScoreFormula{
	"conns_created/s":       {/* contains filtered or unexported fields */},
	"conns_current":         {/* contains filtered or unexported fields */},
	"cpu_idle":              {/* contains filtered or unexported fields */},
	"cpu_iowait":            {/* contains filtered or unexported fields */},
	"cpu_system":            {/* contains filtered or unexported fields */},
	"cpu_user":              {/* contains filtered or unexported fields */},
	"disku_":                {/* contains filtered or unexported fields */},
	"iops_":                 {/* contains filtered or unexported fields */},
	"latency_command":       {/* contains filtered or unexported fields */},
	"latency_read":          {/* contains filtered or unexported fields */},
	"latency_write":         {/* contains filtered or unexported fields */},
	"mem_page_faults":       {/* contains filtered or unexported fields */},
	"mem_resident":          {/* contains filtered or unexported fields */},
	"ops_":                  {/* contains filtered or unexported fields */},
	"queued_read":           {/* contains filtered or unexported fields */},
	"queued_write":          {/* contains filtered or unexported fields */},
	"scan_keys":             {/* contains filtered or unexported fields */},
	"scan_objects":          {/* contains filtered or unexported fields */},
	"scan_sort":             {/* contains filtered or unexported fields */},
	"ticket_avail_read":     {/* contains filtered or unexported fields */},
	"ticket_avail_write":    {/* contains filtered or unexported fields */},
	"wt_cache_used":         {/* contains filtered or unexported fields */},
	"wt_cache_dirty":        {/* contains filtered or unexported fields */},
	"wt_dhandles_active":    {/* contains filtered or unexported fields */},
	"wt_modified_evicted":   {/* contains filtered or unexported fields */},
	"wt_unmodified_evicted": {/* contains filtered or unexported fields */},
}

FormulaMap holds low and high watermarks

Functions

func GetFormulaHTML 

func GetFormulaHTML(metric string) string

GetFormulaHTML returns scoring formula

func GetMetricsFilenames 

func GetMetricsFilenames(filenames []string) []string

GetMetricsFilenames gets metrics or keyhole_stats filesnames

func GetOptime 

func GetOptime(optime interface{}) int64

GetOptime -

func GetScoreByRange 

func GetScoreByRange(v float64, low float64, high float64) int

GetScoreByRange gets score

func GetShortLabel 

func GetShortLabel(label string) string

GetShortLabel gets shorten label

func PrintAllStats 

func PrintAllStats(docs []ServerStatusDoc, span int) string

PrintAllStats print all stats

Types

type Assessment 

type Assessment struct {
	// contains filtered or unexported fields
}

Assessment stores timeserie data

func NewAssessment 

func NewAssessment(stats FTDCStats) *Assessment

NewAssessment returns assessment object

func (*Assessment) GetAssessment 

func (as *Assessment) GetAssessment(from time.Time, to time.Time) map[string]interface{}

GetAssessment gets assessment summary

func (*Assessment) SetVerbose 

func (as *Assessment) SetVerbose(verbose bool)

SetVerbose sets verbose level

type Attribs 

type Attribs struct {
	// contains filtered or unexported fields
}

Attribs stores attribs map

func NewAttribs 

func NewAttribs(attribsMap *map[string][]uint64) *Attribs

NewAttribs returns Attribs structure

func (*Attribs) GetServerStatusDataPoints 

func (attr *Attribs) GetServerStatusDataPoints(i int) ServerStatusDoc

GetServerStatusDataPoints returns server status

func (*Attribs) GetSystemMetricsDataPoints 

func (attr *Attribs) GetSystemMetricsDataPoints(i int) SystemMetricsDoc

GetSystemMetricsDataPoints returns system metrics

type BuildInfo 

type BuildInfo struct {
	Version string `json:"version" bson:"version"`
}

BuildInfo -

type CPUMetrics 

type CPUMetrics struct {
	IdleMS    uint64 `json:"idle_ms" bson:"idle_ms"`
	IOWaitMS  uint64 `json:"iowait_ms" bson:"iowait_ms"`
	NiceMS    uint64 `json:"nice_ms" bson:"nice_ms"`
	SoftirqMS uint64 `json:"softirq_ms" bson:"softirq_ms"`
	StealMS   uint64 `json:"steal_ms" bson:"steal_ms"`
	SystemMS  uint64 `json:"system_ms" bson:"system_ms"`
	UserMS    uint64 `json:"user_ms" bson:"user_ms"`
	TotalMS   uint64
}

CPUMetrics -

type ConcurrentTransactionsCountDoc 

type ConcurrentTransactionsCountDoc struct {
	Available    uint64 `json:"available" bson:"available"`
	Out          uint64 `json:"out" bson:"out"`
	TotalTickets uint64 `json:"totalTickets" bson:"totalTickets"`
}

ConcurrentTransactionsCountDoc contains db.serverStatus().wiredTiger.concurrentTransactions.[read|write]

type ConcurrentTransactionsDoc 

type ConcurrentTransactionsDoc struct {
	Read  ConcurrentTransactionsCountDoc `json:"read" bson:"read"`
	Write ConcurrentTransactionsCountDoc `json:"write" bson:"write"`
}

ConcurrentTransactionsDoc contains db.serverStatus().wiredTiger.concurrentTransactions

type ConnectionsDoc 

type ConnectionsDoc struct {
	Current      uint64 `json:"current" bson:"current"`
	Available    uint64 `json:"available" bson:"available"`
	TotalCreated uint64 `json:"totalCreated" bson:"totalCreated"`
	Active       uint64 `json:"active" bson:"active"`
}

ConnectionsDoc contains db.serverStatus().connections

type DiagnosticData 

type DiagnosticData struct {
	ServerInfo        interface{}
	ServerStatusList  []ServerStatusDoc
	ReplSetStatusList []ReplSetStatusDoc
	SystemMetricsList []SystemMetricsDoc
	// contains filtered or unexported fields
}

DiagnosticData -

func NewDiagnosticData 

func NewDiagnosticData() *DiagnosticData

NewDiagnosticData -

func (*DiagnosticData) AnalyzeServerStatus 

func (d *DiagnosticData) AnalyzeServerStatus(reader *bufio.Reader) error

AnalyzeServerStatus -

func (*DiagnosticData) DecodeDiagnosticData 

func (d *DiagnosticData) DecodeDiagnosticData(filenames []string) error

DecodeDiagnosticData decodes FTDC data files

func (*DiagnosticData) GetEndPoints 

func (d *DiagnosticData) GetEndPoints() []string

GetEndPoints gets grafana uri

type DiagnosticDoc 

type DiagnosticDoc struct {
	Start            time.Time        `json:"start" bson:"start"`
	ServerStatus     ServerStatusDoc  `json:"serverStatus" bson:"serverStatus"`
	ReplSetGetStatus ReplSetStatusDoc `json:"replSetGetStatus" bson:"replSetGetStatus"`
	SystemMetrics    SystemMetricsDoc `json:"systemMetrics" bson:"systemMetrics"`
	End              time.Time        `json:"end" bson:"end"`
}

DiagnosticDoc -

type DiskMetrics 

type DiskMetrics struct {
	ReadTimeMS   uint64 `json:"read_time_ms" bson:"read_time_ms"`
	WriteTimeMS  uint64 `json:"write_time_ms" bson:"write_time_ms"`
	IOInProgress uint64 `json:"io_in_progress" bson:"io_in_progress"`
	IOQueuedMS   uint64 `json:"io_queued_ms" bson:"io_queued_ms"`
	IOTimeMS     uint64 `json:"io_time_ms" bson:"io_time_ms"`
	Reads        uint64 `json:"reads" bson:"reads"`
	Writes       uint64 `json:"writes" bson:"writes"`
}

DiskMetrics -

type DiskStats 

type DiskStats struct {
	IOPS         TimeSeriesDoc
	IOInProgress TimeSeriesDoc
	IOQueuedMS   TimeSeriesDoc
	ReadTimeMS   TimeSeriesDoc
	WriteTimeMS  TimeSeriesDoc
	Utilization  TimeSeriesDoc
}

DiskStats -

type DocumentDoc 

type DocumentDoc struct {
	Deleted  int `json:"deleted" bson:"deleted"`
	Inserted int `json:"inserted" bson:"inserted"`
	Returned int `json:"returned" bson:"returned"`
	Updated  int `json:"updated" bson:"updated"`
}

DocumentDoc contains db.serverStatus().document

type ExtraInfoDoc 

type ExtraInfoDoc struct {
	PageFaults uint64 `json:"page_faults" bson:"page_faults"`
}

ExtraInfoDoc contains db.serverStatus().extra_info

type FTDCStats 

type FTDCStats struct {
	DiskStats         map[string]DiskStats
	MaxWTCache        float64
	ReplicationLags   map[string]TimeSeriesDoc
	ReplSetLegends    []string
	ReplSetStatusList []ReplSetStatusDoc
	ServerInfo        ServerInfoDoc
	ServerStatusList  []ServerStatusDoc
	SystemMetricsList []SystemMetricsDoc
	TimeSeriesData    map[string]TimeSeriesDoc
}

FTDCStats FTDC stats

type GlobalLockDoc 

type GlobalLockDoc struct {
	ActiveClients GlobalLockSubDoc `json:"activeClients" bson:"activeClients"`
	CurrentQueue  GlobalLockSubDoc `json:"currentQueue" bson:"currentQueue"`
	TotalTime     int              `json:"totalTime" bson:"totalTime"`
}

GlobalLockDoc contains db.serverStatus().globalLockDoc

type GlobalLockSubDoc 

type GlobalLockSubDoc struct {
	Readers uint64 `json:"readers" bson:"readers"`
	Total   uint64 `json:"total" bson:"total"`
	Writers uint64 `json:"writers" bson:"writers"`
}

GlobalLockSubDoc contains db.serverStatus().globalLockDoc.[activeClients|currentQueue]

type HostInfo 

type HostInfo struct {
	OS     OSDoc     `json:"os" bson:"os"`
	System SystemDoc `json:"system" bson:"system"`
}

HostInfo -

type MemDoc 

type MemDoc struct {
	Resident uint64 `json:"resident" bson:"resident"`
	Virtual  uint64 `json:"virtual" bson:"virtual"`
}

MemDoc containers db.serverStatus().mem

type MemberDoc 

type MemberDoc struct {
	Name   string      `json:"name" bson:"name"`
	Optime interface{} `json:"optime" bson:"optime"`
	State  int         `json:"state" bson:"state"`
}

MemberDoc stores replset status

type Metrics 

type Metrics struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Metrics stores metrics from FTDC data

func NewMetrics 

func NewMetrics() *Metrics

NewMetrics returns &Metrics

func (*Metrics) AddFTDCDetailStats 

func (m *Metrics) AddFTDCDetailStats(diag *DiagnosticData)

AddFTDCDetailStats assign FTDC values

func (*Metrics) Handler 

func (m *Metrics) Handler(w http.ResponseWriter, r *http.Request)

Handler handle HTTP requests

func (*Metrics) ProcessFiles 

func (m *Metrics) ProcessFiles(filenames []string) error

ProcessFiles reads metrics files/data

func (*Metrics) SetLatest 

func (m *Metrics) SetLatest(latest int)

SetLatest sets latest

func (*Metrics) SetVerbose 

func (m *Metrics) SetVerbose(verbose bool)

SetVerbose sets verbose mode

type MetricsDoc 

type MetricsDoc struct {
	Document      DocumentDoc      `json:"document" bson:"document"`
	QueryExecutor QueryExecutorDoc `json:"queryExecutor" bson:"queryExecutor"`
	Operation     OperationDoc     `json:"operation" bson:"operation"`
}

MetricsDoc contains db.serverStatus().metrics

type NetworkDoc 

type NetworkDoc struct {
	BytesIn          uint64 `json:"bytesIn" bson:"bytesIn"`
	BytesOut         uint64 `json:"bytesOut" bson:"bytesOut"`
	NumRequests      uint64 `json:"numRequests" bson:"numRequests"`
	PhysicalBytesIn  uint64 `json:"physicalBytesIn" bson:"physicalBytesIn"`
	PhysicalBytesOut uint64 `json:"physicalBytesOut" bson:"physicalBytesOut"`
}

NetworkDoc contains db.serverStatus().network

type OSDoc 

type OSDoc struct {
	Name    string `json:"name" bson:"name"`
	Type    string `json:"type" bson:"type"`
	Version string `json:"version" bson:"version"`
}

OSDoc -

type OpCountersDoc 

type OpCountersDoc struct {
	Command uint64 `json:"command" bson:"command"`
	Delete  uint64 `json:"delete" bson:"delete"`
	Getmore uint64 `json:"getmore" bson:"getmore"`
	Insert  uint64 `json:"insert" bson:"insert"`
	Query   uint64 `json:"query" bson:"query"`
	Update  uint64 `json:"update" bson:"update"`
}

OpCountersDoc contains db.serverStatus().OpCounters

type OpLatenciesDoc 

type OpLatenciesDoc struct {
	Commands OpLatenciesOpDoc `json:"commands" bson:"commands"`
	Reads    OpLatenciesOpDoc `json:"reads" bson:"reads"`
	Writes   OpLatenciesOpDoc `json:"writes" bson:"writes"`
}

OpLatenciesDoc contains db.serverStatus().opLatencies

type OpLatenciesOpDoc 

type OpLatenciesOpDoc struct {
	Latency uint64 `json:"latency" bson:"latency"`
	Ops     uint64 `json:"ops" bson:"ops"`
}

OpLatenciesOpDoc contains doc of db.serverStatus().opLatencies

type OperationDoc 

type OperationDoc struct {
	ScanAndOrder   uint64 `json:"scanAndOrder" bson:"scanAndOrder"`
	WriteConflicts uint64 `json:"writeConflicts" bson:"writeConflicts"`
}

OperationDoc contains db.serverStatus().operation

type OptimeDoc 

type OptimeDoc struct {
	T  int64 `json:"t" bson:"t"`
	TS int64 `json:"ts" bson:"ts"`
}

OptimeDoc -

type QueryExecutorDoc 

type QueryExecutorDoc struct {
	Scanned        uint64 `json:"scanned" bson:"scanned"`
	ScannedObjects uint64 `json:"scannedObjects" bson:"scannedObjects"`
}

QueryExecutorDoc contains db.serverStatus().queryExecutor

type QueryRequest 

type QueryRequest struct {
	Timezone string      `json:"timezone"`
	Range    RangeDoc    `json:"range"`
	Targets  []TargetDoc `json:"targets"`
}

QueryRequest -

type RangeDoc 

type RangeDoc struct {
	From time.Time `json:"from"`
	To   time.Time `json:"to"`
}

RangeDoc -

type ReplSetStatusDoc 

type ReplSetStatusDoc struct {
	Date    time.Time   `json:"date" bson:"date"`
	Members []MemberDoc `json:"members" bson:"members"`
}

ReplSetStatusDoc stores replset status

type ScoreFormula 

type ScoreFormula struct {
	// contains filtered or unexported fields
}

ScoreFormula holds metric info

type ServerInfoDoc 

type ServerInfoDoc struct {
	HostInfo  HostInfo  `json:"hostInfo" bson:"hostInfo"`
	BuildInfo BuildInfo `json:"buildInfo" bson:"buildInfo"`
}

ServerInfoDoc -

type ServerStatusDoc 

type ServerStatusDoc struct {
	Connections ConnectionsDoc `json:"connections" bson:"connections"`
	ExtraInfo   ExtraInfoDoc   `json:"extra_info" bson:"extra_info"`
	GlobalLock  GlobalLockDoc  `json:"globalLock" bson:"globalLock"`
	Host        string         `json:"host" bson:"host"`
	LocalTime   time.Time      `json:"localTime" bson:"localTime"`
	Mem         MemDoc         `json:"mem" bson:"mem"`
	Metrics     MetricsDoc     `json:"metrics" bson:"metrics"`
	Network     NetworkDoc     `json:"network" bson:"network"`
	OpCounters  OpCountersDoc  `json:"opcounters" bson:"opcounters"`
	OpLatencies OpLatenciesDoc `json:"opLatencies" bson:"opLatencies"`
	Process     string         `json:"process" bson:"process"`
	Repl        bson.M         `json:"repl" bson:"repl"`
	Sharding    bson.M         `json:"sharding" bson:"sharding"`
	Uptime      uint64         `json:"uptime" bson:"uptime"`
	Version     string         `json:"version" bson:"version"`
	WiredTiger  WiredTigerDoc  `json:"wiredTiger" bson:"wiredTiger"`
}

ServerStatusDoc contains docs from db.serverStatus()

type SystemDoc 

type SystemDoc struct {
	CPUArch   string `json:"cpuArch" bson:"cpuArch"`
	Hostname  string `json:"hostname" bson:"hostname"`
	NumCores  int    `json:"numCores" bson:"numCores"`
	MemSizeMB int    `json:"memSizeMB" bson:"memSizeMB"`
}

SystemDoc -

type SystemMetricsDoc 

type SystemMetricsDoc struct {
	Start time.Time              `json:"start" bson:"start"`
	CPU   CPUMetrics             `json:"cpu" bson:"cpu"`
	Disks map[string]DiskMetrics `json:"disks" bson:"disks"`
}

SystemMetricsDoc -

type TargetDoc 

type TargetDoc struct {
	Target string `json:"target"`
	RefID  string `json:"refId"`
	Type   string `json:"type"`
}

TargetDoc -

type TimeSeriesDoc 

type TimeSeriesDoc struct {
	Target     string      `json:"target"`
	DataPoints [][]float64 `json:"datapoints"`
}

TimeSeriesDoc -

func FilterTimeSeriesData 

func FilterTimeSeriesData(tsData TimeSeriesDoc, from time.Time, to time.Time) TimeSeriesDoc

FilterTimeSeriesData returns partial data points if there are too many

type WiredTigerBlockManagerDoc 

type WiredTigerBlockManagerDoc struct {
	BytesRead              uint64 `json:"bytes read"`
	BytesWritten           uint64 `json:"bytes written"`
	BytesWrittenCheckPoint uint64 `json:"bytes written for checkpoint"`
}

WiredTigerBlockManagerDoc contains db.serverStatus().wiredTiger.cache

type WiredTigerCacheDoc 

type WiredTigerCacheDoc struct {
	BytesReadIntoCache     uint64 `json:"bytes read into cache" bson:"bytes read into cache"`
	BytesWrittenFromCache  uint64 `json:"bytes written from cache" bson:"bytes written from cache"`
	CurrentlyInCache       uint64 `json:"bytes currently in the cache" bson:"bytes currently in the cache"`
	MaxBytesConfigured     uint64 `json:"maximum bytes configured" bson:"maximum bytes configured"`
	ModifiedPagesEvicted   uint64 `json:"modified pages evicted" bson:"modified pages evicted"`
	UnmodifiedPagesEvicted uint64 `json:"unmodified pages evicted" bson:"unmodified pages evicted"`
	TrackedDirtyBytes      uint64 `json:"tracked dirty bytes in the cache" bson:"tracked dirty bytes in the cache"`
}

WiredTigerCacheDoc contains db.serverStatus().wiredTiger.cache

type WiredTigerDataHandleDoc 

type WiredTigerDataHandleDoc struct {
	Active uint64 `json:"connection data handles currently active" bson:"connection data handles currently active"`
	Size   uint64 `json:"connection data handle size" bson:"connection data handle size"`
}

WiredTigerDataHandleDoc contains db.serverStatus().wiredTiger.data-handle

type WiredTigerDoc 

type WiredTigerDoc struct {
	Perf                   interface{}               `json:"perf" bson:"perf"`
	BlockManager           WiredTigerBlockManagerDoc `json:"block-manager" bson:"block-manager"`
	Cache                  WiredTigerCacheDoc        `json:"cache" bson:"cache"`
	ConcurrentTransactions ConcurrentTransactionsDoc `json:"concurrentTransactions" bson:"concurrentTransactions"`
	DataHandle             WiredTigerDataHandleDoc   `json:"data-handle" bson:"data-handle"`
}

WiredTigerDoc contains db.serverStatus().wiredTiger

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.