stat

package
v0.0.0-...-1dd1f65 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 26, 2023 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package stat implements c4t's persistent statistics support.

This includes the models for statistics collection (Set, MachineSet, etc), and the Persister, a director observer that tracks statistics by persisting them to a JSON file.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func OpenStatFile 

func OpenStatFile(name string) (*os.File, error)

OpenStatFile opens a file in the appropriate mode for using it as a statistics target.

Types

type Hitset 

type Hitset struct {
	// Timespan records the first and most recent times this mutant was hit in this way.
	Timespan timing.Span `json:"time_span,omitempty"`
	// Count is the number of times this mutant was hit in this way.
	Count uint64 `json:"count,omitempty"`
}

Hitset is a set of statistics relating to the way in which a mutant has been 'hit'.

func (*Hitset) Add 

func (h *Hitset) Add(ntimes uint64, ts timing.Span)

Add records ntimes hits over timespan ts.

func (*Hitset) AtLeastOnce 

func (h *Hitset) AtLeastOnce() bool

AtLeastOnce gets whether the mutant was hit in a particular way at least once.

type Machine 

type Machine struct {
	// LastCycle is the last announced cycle in this session.
	LastCycle director.Cycle `json:"last_cycle,omitempty"`

	// Session contains statistics for this session.
	Session MachineSpan `json:"session,omitempty"`
	// Total contains statistics across all sessions.
	Total MachineSpan `json:"total,omitempty"`
}

Machine is a statistics set for a specific machine.

func (*Machine) AddAnalysis 

func (m *Machine) AddAnalysis(a analysis.Analysis)

AddAnalysis adds the information from analysis a to this machine statset.

func (*Machine) AddCycle 

func (m *Machine) AddCycle(c director.CycleMessage)

AddCycle adds the information from cycle message c to this machine statset.

func (*Machine) DumpMutationCSV 

func (m *Machine) DumpMutationCSV(w *csv.Writer, machine id.ID, total bool) error

DumpMutationCSV dumps into w a CSV representation of the mutation statistics in this machine. Each line in the record has mid as a prefix. If total is true, the multi-session totals will be dumped; otherwise, this session's totals will be dumped. The writer is flushed at the end of this dump.

func (*Machine) ResetForSession 

func (m *Machine) ResetForSession()

ResetForSession removes from this statset any statistics that no longer apply across session boundaries.

type MachineSpan 

type MachineSpan struct {
	// FinishedCycles counts the number of cycles that finished.
	FinishedCycles uint64 `json:"finished_cycles"`
	// ErroredCycles counts the number of cycles that resulted in an error.
	ErroredCycles uint64 `json:"errored_cycles"`
	// Mutation contains totals for mutation testing since this span started.
	Mutation Mutation `json:"mutation,omitempty"`

	// SessionStatusTotals contains status totals since this span started.
	// It may be empty if this machine has not yet been active this span.
	StatusTotals map[status.Status]uint64 `json:"status_totals,omitempty"`
}

MachineSpan contains the timespan-specific part of Machine.

func (*MachineSpan) AddAnalysis 

func (m *MachineSpan) AddAnalysis(a analysis.Analysis)

AddAnalysis adds the information from analysis a to this machine statset.

func (*MachineSpan) AddCycle 

func (m *MachineSpan) AddCycle(c director.CycleMessage)

AddCycle adds the information from cycle message c to this machine span.

func (*MachineSpan) DumpMutationCSV 

func (m *MachineSpan) DumpMutationCSV(w *csv.Writer, machine id.ID) error

DumpMutationCSV dumps into w a CSV representation of the mutation statistics in this machine span. Each line in the record has mid as a prefix. The writer is flushed at the end of this dump.

func (*MachineSpan) Reset 

func (m *MachineSpan) Reset()

Reset resets a machine span.

type Mutant 

type Mutant struct {
	// Info contains the full mutant metadata set for the mutant.
	Info mutation.Mutant `json:"info,omitempty"`
	// Selections records the number of times this mutant has been selected.
	Selections Hitset `json:"selections,omitempty"`
	// Hits records the number of times this mutant has been hit (including kills).
	Hits Hitset `json:"hits,omitempty"`
	// Kills records the number of selections that resulted in kills.
	Kills Hitset `json:"kills,omitempty"`
	// Statuses records, for each status, the number of selections that resulted in that status.
	Statuses map[status.Status]uint64 `json:"statuses,omitempty"`
}

Mutant gives statistics for a particular mutant.

type MutantFilter 

type MutantFilter func(m Mutant) bool

MutantFilter is the type of mutant filtering predicates. 

var (
	// FilterAllMutants is a mutant filter that allows all mutants.
	FilterAllMutants MutantFilter = func(mutant Mutant) bool { return true }
	// FilterHitMutants is a mutant filter that allows hit mutants only.
	FilterHitMutants MutantFilter = func(mutant Mutant) bool { return mutant.Hits.AtLeastOnce() }
	// FilterKilledMutants is a mutant filter that allows killed mutants only.
	FilterKilledMutants MutantFilter = func(mutant Mutant) bool { return mutant.Kills.AtLeastOnce() }
	// FilterEscapedMutants is a mutant filter that allows only mutants that were hit but not killed.
	FilterEscapedMutants MutantFilter = func(mutant Mutant) bool {
		return mutant.Hits.AtLeastOnce() && !mutant.Kills.AtLeastOnce()
	}
)

type Mutation 

type Mutation struct {
	// ByIndex records statsets for each mutant index.
	ByIndex map[mutation.Index]Mutant `json:"by_index"`
}

Mutation holds statistics for each mutant in a mutation testing campaign.

func (*Mutation) AddAnalysis 

func (m *Mutation) AddAnalysis(a mutation.Analysis)

AddAnalysis adds the information from mutation analysis a to this statset.

Example

ExampleMutation_AddAnalysis is a runnable example for AddAnalysis. 

package main

import (
	"fmt"

	"github.com/c4-project/c4t/internal/stat"

	"github.com/c4-project/c4t/internal/subject/status"

	"github.com/c4-project/c4t/internal/id"
	"github.com/c4-project/c4t/internal/mutation"
	"github.com/c4-project/c4t/internal/subject/compilation"
)

func main() {
	var s stat.Mutation
	s.AddAnalysis(mutation.Analysis{
		27: mutation.MutantAnalysis{
			Mutant: mutation.NamedMutant(27, "ABC", 1),
			Selections: []mutation.SelectionAnalysis{
				{
					NumHits: 0,
					Status:  status.Ok,
					HitBy:   compilation.Name{SubjectName: "smooth", CompilerID: id.FromString("criminal")},
				},
				{
					NumHits: 2,
					Status:  status.RunFail,
					HitBy:   compilation.Name{SubjectName: "marco", CompilerID: id.FromString("polo")},
				},
				{
					NumHits: 4,
					Status:  status.Flagged,
					HitBy:   compilation.Name{SubjectName: "mint", CompilerID: id.FromString("polo")},
				},
			},
		},
		53: mutation.MutantAnalysis{
			Mutant: mutation.NamedMutant(53, "DEF", 0),
			Selections: []mutation.SelectionAnalysis{
				{
					NumHits: 0,
					Status:  status.Filtered,
					HitBy:   compilation.Name{SubjectName: "marco", CompilerID: id.FromString("polo")},
				},
			},
		},
	})

	fmt.Println(s.ByIndex[27].Info, "selected:", s.ByIndex[27].Selections.Count, "hit:", s.ByIndex[27].Hits.Count, "killed:", s.ByIndex[27].Kills.Count)
	fmt.Println(s.ByIndex[53].Info, "selected:", s.ByIndex[53].Selections.Count, "hit:", s.ByIndex[53].Hits.Count, "killed:", s.ByIndex[53].Kills.Count)

}

Output:

ABC1:27 selected: 3 hit: 6 killed: 1
DEF:53 selected: 1 hit: 0 killed: 0

func (*Mutation) DumpCSV 

func (m *Mutation) DumpCSV(w *csv.Writer, machine id.ID) error

DumpCSV dumps into w a CSV representation of this mutation statistics set. Each line in the record has machine as a prefix. The writer is flushed at the end of this dump.

Example

ExampleMutation_DumpCSV is a runnable example for Mutation.DumpCSV. 

package main

import (
	"encoding/csv"
	"os"

	"github.com/c4-project/c4t/internal/stat"

	"github.com/c4-project/c4t/internal/subject/status"

	"github.com/c4-project/c4t/internal/id"
	"github.com/c4-project/c4t/internal/mutation"
)

func main() {
	_ = (&stat.Mutation{
		ByIndex: map[mutation.Index]stat.Mutant{
			2:  {Info: mutation.AnonMutant(2), Selections: stat.Hitset{Count: 1}, Hits: stat.Hitset{Count: 0}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Filtered: 1}},
			42: {Info: mutation.NamedMutant(42, "FOO", 0), Selections: stat.Hitset{Count: 10}, Hits: stat.Hitset{Count: 1}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Ok: 9, status.CompileTimeout: 1}},
			53: {Info: mutation.NamedMutant(53, "BAR", 10), Selections: stat.Hitset{Count: 20}, Hits: stat.Hitset{Count: 400}, Kills: stat.Hitset{Count: 15}, Statuses: map[status.Status]uint64{status.Flagged: 15, status.CompileFail: 3, status.RunFail: 2}},
		},
	}).DumpCSV(csv.NewWriter(os.Stdout), id.FromString("localhost"))

}

Output:

localhost,2,,1,0,0,0,1,0,0,0,0,0
localhost,42,FOO,10,1,0,9,0,0,0,1,0,0
localhost,53,BAR10,20,400,15,0,0,15,3,0,2,0

func (*Mutation) KilledMutants 

func (m *Mutation) KilledMutants() []mutation.Mutant

KilledMutants returns a sorted list of all mutant IDs killed in this statset.

func (*Mutation) Mutants 

func (m *Mutation) Mutants() []mutation.Mutant

Mutants returns a sorted list of all mutant IDs seen in this statset.

func (Mutation) MutantsWhere 

func (m Mutation) MutantsWhere(pred func(m Mutant) bool) []mutation.Mutant

MutantsWhere returns a sorted list of mutants satisfying pred. (It is a value receiver method to allow calling through templates.)

func (*Mutation) Reset 

func (m *Mutation) Reset()

Reset resets all of this statset's maps to empty, but non-nil.

Example

ExampleMutation_Reset is a runnable example for Mutation.Reset. 

package main

import (
	"fmt"

	"github.com/c4-project/c4t/internal/stat"
)

func main() {
	var s stat.Mutation
	s.Reset()

	fmt.Println("by-mutant nil:", s.ByIndex == nil, "len:", len(s.ByIndex))

}

Output:

by-mutant nil: false len: 0

type Persister 

type Persister struct {
	// contains filtered or unexported fields
}

Persister is a forward handler that maintains and persists a statistics set on disk.

func NewPersister 

func NewPersister(f *os.File) (*Persister, error)

NewPersister creates a Persister that reads and writes statistics from f. If f is non-empty, it immediately tries to read any existing stats dump, and fails if this doesn't work. The Persister takes ownership of f; close f with the Persister's Close method.

func (*Persister) Close 

func (s *Persister) Close() error

Close closes this Persister, returning any errors arising from either the stats persisting or file close.

func (*Persister) OnCycle 

func (s *Persister) OnCycle(c director.CycleMessage)

OnCycle feeds the information from c into the stats set.

func (*Persister) OnCycleAnalysis 

func (s *Persister) OnCycleAnalysis(a director.CycleAnalysis)

OnCycleAnalysis feeds the information from a into the stats set.

func (*Persister) OnCycleBuild 

func (s *Persister) OnCycleBuild(c director.Cycle, m builder.Message)

OnCycleBuild feeds the information from c and m into the stats set.

func (*Persister) OnCycleCompiler 

func (s *Persister) OnCycleCompiler(c director.Cycle, m compiler.Message)

OnCycleCompiler feeds the information from c and m into the stats set.

func (*Persister) OnCycleCopy 

func (s *Persister) OnCycleCopy(c director.Cycle, m copier.Message)

OnCycleCopy feeds the information from c and m into the stats set.

func (*Persister) OnCycleInstance 

func (s *Persister) OnCycleInstance(c director.Cycle, m director.InstanceMessage)

OnCycleInstance feeds the information from c and m into the stats set.

func (*Persister) OnCycleSave 

func (s *Persister) OnCycleSave(c director.Cycle, m saver.ArchiveMessage)

OnCycleSave feeds the information from c and m into the stats set.

func (*Persister) OnMachines 

func (s *Persister) OnMachines(m machine.Message)

OnMachines feeds the information from m into the stats set.

func (*Persister) OnPrepare 

func (s *Persister) OnPrepare(m director.PrepareMessage)

OnPrepare feeds the information from m into the stats set.

type Set 

type Set struct {

	// StartTime is the time at which this statistics set was first created.
	StartTime time.Time `json:"start_time,omitempty"`

	// EventCount is the number of events that have been applied to this Set.
	// This metric isn't particularly exciting from a user perspective, but is used to prevent spurious disk flushes.
	EventCount uint64 `json:"event_count"`

	// SessionStartTime is the time at which this session started (ie, the statset was last reloaded from disk).
	SessionStartTime time.Time `json:"session_start_time,omitempty"`

	// Machines is a map from machine IDs to statistics about those machines.
	Machines map[id.ID]Machine `json:"machines,omitempty"`
}

Set aggregates statistics taken from cycle analyses.

func (*Set) DumpMutationCSV 

func (s *Set) DumpMutationCSV(w *csv.Writer, total bool) error

DumpMutationCSV dumps into w a CSV representation of the mutation statistics in this set. Each machine record has its lines prefixed by its machine ID, is flushed separately, and appears in ID order. If total is true, the multi-session totals will be dumped; otherwise, this session's totals will be dumped.

Example

ExampleSet_DumpMutationCSV is a runnable example for Set.DumpMutationCSV. 

package main

import (
	"encoding/csv"
	"fmt"
	"os"

	"github.com/c4-project/c4t/internal/id"

	"github.com/c4-project/c4t/internal/subject/status"

	"github.com/c4-project/c4t/internal/mutation"
	"github.com/c4-project/c4t/internal/stat"
)

func main() {
	s := stat.Set{
		Machines: map[id.ID]stat.Machine{
			id.FromString("foo"): {
				Session: stat.MachineSpan{
					Mutation: stat.Mutation{
						ByIndex: map[mutation.Index]stat.Mutant{
							2:  {Info: mutation.AnonMutant(2), Selections: stat.Hitset{Count: 1}, Hits: stat.Hitset{Count: 0}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Filtered: 1}},
							42: {Info: mutation.NamedMutant(42, "FOO", 0), Selections: stat.Hitset{Count: 10}, Hits: stat.Hitset{Count: 1}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Ok: 9, status.CompileTimeout: 1}},
							53: {Info: mutation.NamedMutant(53, "BAR", 5), Selections: stat.Hitset{Count: 20}, Hits: stat.Hitset{Count: 400}, Kills: stat.Hitset{Count: 15}, Statuses: map[status.Status]uint64{status.Flagged: 15, status.CompileFail: 3, status.RunFail: 2}},
						},
					},
				},
				Total: stat.MachineSpan{
					Mutation: stat.Mutation{
						ByIndex: map[mutation.Index]stat.Mutant{
							2:  {Info: mutation.AnonMutant(2), Selections: stat.Hitset{Count: 41}, Hits: stat.Hitset{Count: 5000}, Kills: stat.Hitset{Count: 40}, Statuses: map[status.Status]uint64{status.Flagged: 40, status.Filtered: 1}},
							42: {Info: mutation.NamedMutant(42, "FOO", 0), Selections: stat.Hitset{Count: 100}, Hits: stat.Hitset{Count: 1}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Ok: 99, status.CompileTimeout: 1}},
							53: {Info: mutation.NamedMutant(53, "BAR", 5), Selections: stat.Hitset{Count: 20}, Hits: stat.Hitset{Count: 400}, Kills: stat.Hitset{Count: 15}, Statuses: map[status.Status]uint64{status.Flagged: 15, status.CompileFail: 3, status.RunFail: 2}},
						},
					},
				},
			},
			id.FromString("bar"): {
				Total: stat.MachineSpan{
					Mutation: stat.Mutation{
						ByIndex: map[mutation.Index]stat.Mutant{
							1: {Info: mutation.AnonMutant(1), Selections: stat.Hitset{Count: 500}, Hits: stat.Hitset{Count: 0}, Kills: stat.Hitset{Count: 0}, Statuses: map[status.Status]uint64{status.Ok: 500}},
						},
					},
				},
			},
		},
	}

	w := csv.NewWriter(os.Stdout)
	_ = s.DumpMutationCSVHeader(w)
	_ = s.DumpMutationCSV(w, false)
	fmt.Println("--")
	_ = s.DumpMutationCSV(w, true)

}

Output:

Machine,Index,Name,Selections,Hits,Kills,Ok,Filtered,Flagged,CompileFail,CompileTimeout,RunFail,RunTimeout
foo,2,,1,0,0,0,1,0,0,0,0,0
foo,42,FOO,10,1,0,9,0,0,0,1,0,0
foo,53,BAR5,20,400,15,0,0,15,3,0,2,0
--
bar,1,,500,0,0,500,0,0,0,0,0,0
foo,2,,41,5000,40,0,1,40,0,0,0,0
foo,42,FOO,100,1,0,99,0,0,0,1,0,0
foo,53,BAR5,20,400,15,0,0,15,3,0,2,0

func (*Set) DumpMutationCSVHeader 

func (s *Set) DumpMutationCSVHeader(w *csv.Writer) error

DumpMutationCSVHeader dumps into w a CSV header for mutation analysis.

func (*Set) Init 

func (s *Set) Init()

Init initialises statistics in s that should be set when creating a stats file.

func (*Set) Load 

func (s *Set) Load(r io.Reader) error

Load loads stats from r into this Set.

func (*Set) LoadFile 

func (s *Set) LoadFile(name string) error

LoadFile loads stats from filename name into this Set.

For persisting into a Set, use OpenStatFile and NewPersister.

func (*Set) OnCycle 

func (s *Set) OnCycle(c director.CycleMessage)

OnCycle incorporates cycle information from c into the statistics set.

func (*Set) OnCycleAnalysis 

func (s *Set) OnCycleAnalysis(a director.CycleAnalysis)

OnCycleAnalysis incorporates cycle analysis from a into the statistics set.

func (*Set) OnCycleBuild 

func (s *Set) OnCycleBuild(director.Cycle, builder.Message)

OnCycleBuild does nothing, for now.

func (*Set) OnCycleCompiler 

func (s *Set) OnCycleCompiler(director.Cycle, compiler.Message)

OnCycleCompiler does nothing, for now.

func (*Set) OnCycleCopy 

func (s *Set) OnCycleCopy(director.Cycle, copier.Message)

OnCycleCopy does nothing, for now.

func (*Set) OnCycleInstance 

func (s *Set) OnCycleInstance(director.Cycle, director.InstanceMessage)

OnCycleInstance does nothing, for now.

func (*Set) OnCycleSave 

func (s *Set) OnCycleSave(director.Cycle, saver.ArchiveMessage)

OnCycleSave does nothing, for now.

func (*Set) OnMachines 

func (s *Set) OnMachines(machine.Message)

OnMachines does nothing, for now.

func (*Set) OnPrepare 

func (s *Set) OnPrepare(director.PrepareMessage)

OnPrepare does nothing, for now.

func (*Set) ResetForSession 

func (s *Set) ResetForSession()

ResetForSession resets statistics in s that are session-specific.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.