cptest

package module
v1.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 5, 2021 License: BSD-3-Clause Imports: 14 Imported by: 0

README

cptest

Coverage Status GoReport PkgGoDev Actions Status Release

Copy all example test cases from a problem statement into a file. Then test your code in one command!

Hmm?

Let's assume that you have a shortcut for compiling your code. Also the directory with your code looks like this

$ ls
app* Makefile main.cpp

Create a file, say, inputs.txt and copy the test cases in the following format:

input: test 1
---
output for test 1
===
input: test 2
---
output for test 2

That is, just separate input and output with --- and test cases themselves with ===.

Now, simply run in the directory

$ cptest app
using default time limit: 6s
=== RUN Test 1
=== RUN Test 2
=== RUN Test 3
--- WA: Test 1 (0.005s)
Input:
input: test 1

Answer:
output for test 1

Output:
no

--- WA: Test 2 (0.006s)
Input:
input: test 2

Answer:
output for test 2

Output:
no

(Your code was determining if the given string was a palindrome, by the way.)

You can also explicitly provide the working directory and/or the path to the inputs/executable. In general

Usage:
    cptest [-i INPUTS] EXECUTABLE

You can simply amend the command you're executing with the shortcut and that's it! No more risking getting a WA on the first test or wasting time rechecking the given examples!

Time limit

You can also specify a time limit for the tests. Instead of the first test, you can provide a set of key-value pairs. Only one key is supported for now, though -- tl. (If you have any ideas for more, an issue is welcome!)

Assuming, the task is to compute fibonacci numbers, somebody wrote a O(2^n) implementation. This inputs.txt will fail:

tl = 10.0
===
1
---
1
===
2
---
1
===
47
---
2971215073

Running cptest, we get

$ cptest app
=== RUN	Test 1
=== RUN	Test 2
=== RUN	Test 3
--- OK:	Test 1 (0.006s)
--- OK:	Test 2 (0.006s)
--- TL:	Test 3 (10.000s)
Input:
47

Answer:
2971215073

FAIL
2/3 passed

Build

To build and run the application, do (assuming you've cloned the repository)

$ cd cptest
$ go build ./cmd/cptest
$ ./cptest

You can add it to your PATH to call it from anywhere in the console. On Linux run

# mv cptest /usr/bin

Documentation

Index

Constants

View Source 
const (
	STRXM lexemeType = iota
	FLOATXM
	INTXM
	FINALXM
)

Available lexeme types. The following invariant holds: each type is a specialization of all types whose numerical value is less than that of self. For example, 42 is a float and is an int, but 42.2 is a float but not an int. Hence, int is a specialization of float. A type T is a specialization of a type U if any value of type T is of type U also.

The consequence is that between any two types T and U from the list there's always a specialization relationship, but in gerenal, this is not the case. For example: imagine a lexeme type 'hash' that classifies strings of form 2400f9b. The float is not a specialization of hash, because 42.2 is not a hash, and likewise the hash is not a specialization of float, because 2400f9b is not a float.

View Source 
const (
	IOSeparatorMissing = InputsError("IO separator missing")
	KeyMissing         = InputsError("key cannot be empty")
)

A set of errors that may be produced during scanning of the inputs file. These replace sentinel errors making the errors be comparable with equals operator.

View Source 
const (
	IODelim   = "---"
	TestDelim = "==="
)

The set of delimeters used when partitioning inputs file.

View Source 
const AltLineFeed = "\\n"

AltLineFeed is the representation of LF in textual form, that replaces LF when it's colorized.

Variables

View Source 
var Au aurora.Aurora

Au is used to colorize output of several functions. A user of the library can change its value to disable colored output. Refer to the aurora readme for that.

View Source 
var DefaultPrecision uint = 6

DefaultPrecision is the value the TestingBatch lexer's precision is initialized with in NewTestingBatch function.

View Source 
var MaskGenerators = []func(*Lexer, string, string) []bool{
	(*Lexer).GenMaskForString,
	(*Lexer).GenMaskForFloat,
	(*Lexer).GenMaskForInt,
}

MaskGenerators lists all of the mask generating functions (MGF). MGFs are defined only for arguments of the same type. I.e., there's no MGF for float and int, only for float/float, and int/int. If differnt types must be assessed, the MGF of their common type_ must be called. The common type between two types T and U exists if specialization relationship between them exists and is the least specialized type. The index of MGF in this array corresponds to the numerical value of the type of which MGF's arguments are.

View Source 
var TypeCheckers = []func(string) bool{
	IsFloatLexeme,
	IsIntLexeme,
}

TypeCheckers defines a list type checking functions (TCF). The type checker for string is omitted because it always returns true. Hence, the index of TCF corresponds to a type of numerical value `index+1`.

View Source 
var VALID_INT_MAX_LEN = 10

VALID_INT_MAX_LEN is maximum number of digits a lexeme may have to be considered an int

Functions

func AssertCallCount 

func AssertCallCount(t *testing.T, got, want int)

AssertCallCount checks that the received and expected number of calls are equal.

func AssertConfig 

func AssertConfig(t *testing.T, got, want map[string]string)

AssertConfig checks whether received and expected config key-value sets are equal.

func AssertDiffFailure added in v0.2.0 

func AssertDiffFailure(t *testing.T, ok bool)

AssertDiffSuccess chacks if lexeme comparison returned ok = true.

func AssertDiffSuccess added in v0.2.0 

func AssertDiffSuccess(t *testing.T, ok bool)

AssertDiffSuccess chacks if lexeme comparison returned ok = true.

func AssertEnrichedLexSequence added in v0.2.0 

func AssertEnrichedLexSequence(t *testing.T, got, want []RichText)

func AssertErrorLines 

func AssertErrorLines(t *testing.T, errs []error, lines []int)

AssertErrorLines checks that each error in the received array of errors is wrapping a LinedError error. At the same time, it checks that the line numbers are equal to the expected ones.

func AssertErrors 

func AssertErrors(t *testing.T, got, want []error)

AssertErrors compared received array of errors with the expected one.

func AssertLexemes added in v0.2.0 

func AssertLexemes(t *testing.T, got, want []string)

AssertLexSequence compares if the two LexSequences are equal.

func AssertNoConfig 

func AssertNoConfig(t *testing.T, got map[string]string)

AssertNoConfig checks that the received key-value set is empty. If it's not, the test is failed and the its contents are printed.

func AssertNoErrors 

func AssertNoErrors(t *testing.T, errs []error)

AssertNoErrors will check if the array of errors is empty. If it's not empty, the test will be failed and the errors will be reported.

func AssertRichText added in v0.2.0 

func AssertRichText(t *testing.T, got, want RichText)

func AssertRichTextMask added in v0.2.0 

func AssertRichTextMask(t *testing.T, got, want []bool)

func AssertTest 

func AssertTest(t *testing.T, got Test, want Test)

AssertTest compare the inputs and outputs with respective expected ones for equivalence.

func AssertTests 

func AssertTests(t *testing.T, got []Test, want []Test)

AssertTests will compare received array of tests with the expected one.

func AssertText added in v0.2.0 

func AssertText(t *testing.T, got, want string)

func AssertTimes 

func AssertTimes(t *testing.T, got, want map[int]time.Duration)

AssertTimes check whether the received and expected timestampts for the test cases both exist and are equal.

func AssertVerdicts 

func AssertVerdicts(t *testing.T, got, want map[int]Verdict)

AssertVerdicts checks that received and expected verdict maps contain the same keys, and then checks that the values for these keys equal.

func DeduceLexemeType added in v0.2.0 

func DeduceLexemeType(xm string) lexemeType

DeduceLexemeType will assess the type of the lexeme by sequentially applying more and more specialized type checkers starting from the least restrictive one.

func DumpLexemes added in v0.2.0 

func DumpLexemes(xms []RichText, color aurora.Color) string

DumpLexemes is used to transform array of possibly colorized lexemes into a human readable format. The lexemes are separated by spaces. There are no trailing spaces. Colorized newlines are replaced by printable AltLineFeed string.

func IsFloatLexeme added in v0.2.0 

func IsFloatLexeme(xm string) bool

IsFloatLexeme returns true if the string represents a floating-point value. Although, there can be no floating-point inside of it. A floating-point value is of form int_part['.' ('0'-'9')*]

func IsIntLexeme added in v0.2.0 

func IsIntLexeme(xm string) bool

IsIntLexeme returns true if the string represents a signed integer. Additionally, it should contain not more than VALID_INT_MAX_LEN digits.

func ScanConfig 

func ScanConfig(text string) (m map[string]string, errs []error)

ScanConfig tries to parse a stream of key-value pairs. It expects each pair to be located on a dedicated line. Duplicate keys are allowed, the later version is preferred.

func ScanKeyValuePair 

func ScanKeyValuePair(line string) (string, string, error)

ScanKeyValuePair parses the key-value pair of form 'key=value'. Strings without assignment are treated as keys with empty value. Strings with assignment but with empty key are erroneous. The space around key and value respectively is trimmed.

func ScanLexemes added in v0.2.0 

func ScanLexemes(data []byte, atEOF bool) (advance int, token []byte, err error)

ScanLexemes is a split function for bufio.Scanner. It is same as bufio.ScanWords, except that it treats \n character in a special way. \n cannot be in any lexeme, except for "\n" itself. Hence, several \n\n are parsed as separate lexemes ("\n", "\n"). It will never return an empty lexeme. The definition of other spaces is set by unicode.IsSpace.

func SplitByInlinedPrefixN 

func SplitByInlinedPrefixN(text, delim string, n int) (parts []string)

SplitByInlinedPrefixN works in the same way as strings.SplitN. However, it does one additional thing. It matches the *prefixes of the lines* for equality with the delimeter. Upon match the entire line is discarded.

If text doesn't contain the delimeter, only one part is returned. User can specify the number of parts they want at most via the third argument.

func TestEndStub added in v0.2.0 

func TestEndStub(b *TestingBatch, test Test, id int)

TestEndStub is a stub for TestEndCallback that does nothing.

func TestStartStub added in v0.2.0 

func TestStartStub(id int)

TestStartStub is a stub for TestStartCallback that does nothing.

Types

type ConfigurableStopwatcher 

type ConfigurableStopwatcher struct {
	// contains filtered or unexported fields
}

ConfigurableStopwatcher is an implementation of the Stopwatcher that uses real time.

func NewConfigurableStopwatcher 

func NewConfigurableStopwatcher(TL time.Duration) *ConfigurableStopwatcher

NewConfigurableStopwatcher will return an initialized ConfigurableStopwatcher with the desired time limit. If time limit specified is zero or negative, the time limit will never fire.

func (*ConfigurableStopwatcher) Elapsed 

func (s *ConfigurableStopwatcher) Elapsed() time.Duration

Elapsed returns the true number of seconds since the initialization of the ConfigurableStopwatcher.

func (*ConfigurableStopwatcher) TimeLimit 

func (s *ConfigurableStopwatcher) TimeLimit() <-chan time.Duration

TimeLimit returns a channel that will send back the number of seconds passed since beginning until the time limit was fired. The returned value may not equal to the time limit ConfigurableStopwatcher was initialized with.

type Inputs 

type Inputs struct {
	Tests  []Test
	Config map[string]string
}

Inputs contains all information located in the inputs file: tests and the set of key-value pairs that were provided.

func ScanInputs 

func ScanInputs(text string) (inputs Inputs, errs []error)

ScanInputs is the main routine for parsing inputs file. It splits the input by test case separator, and tries to parse each individual test case one by one. If the first meaningful test could not be parsed without errors, it is interpreted as a configuration and parsed again. There may be multiple configurations per file. The later ones overwrite keys set by former ones. The empty tests are skipped (those that don't contain input, output and the separator). If test case could not be parsed, parsing continues to the next test case, but the errors are accumulated and returned together.

type InputsError 

type InputsError string

InputsError represents an error produced while scanning inputs file.

func (InputsError) Error 

func (e InputsError) Error() string

type Lexer added in v0.2.0 

type Lexer struct {
	Precision uint
}

Lexer is a set of settings that control lexeme scanning and comparison. And the methods for scanning and comparison are conviniently methods of Lexer.

func (*Lexer) Compare added in v0.2.0 

func (l *Lexer) Compare(target, source []string) (rts []RichText, ok bool)

Compare compares target against source and generates colored target's lexems highlighting mismatches between them. Additionally, actual comparison takes place between two non-LF lexems, and the spurious LFs are marked red and skipped. The function is intended to be called twice for the two permutations of the arguments to get error highlighting for both strings.

func (*Lexer) GenMaskForFloat added in v0.2.0 

func (l *Lexer) GenMaskForFloat(target, source string) (mask []bool)

GenMaskForFloat uses the same logic as GenMaskForInt to highlight the whole part. If at least one digit in the fractional part (part after the dot) is different and its index (zero-based) is less than lexer's precision, this digit is highlighted.

func (*Lexer) GenMaskForInt added in v0.2.0 

func (l *Lexer) GenMaskForInt(target, source string) (mask []bool)

GenMaskForInt will highlight the whole number if at least one digit is different. Independently, the sign will be highlighted if it's different also.

func (*Lexer) GenMaskForString added in v0.2.0 

func (l *Lexer) GenMaskForString(target, source string) (mask []bool)

GenMaskForString will highlight mismatching characters.

func (*Lexer) GenerateMask added in v0.2.0 

func (l *Lexer) GenerateMask(target, source string) []bool

GenerateMask is a wrapper function that finds the common type of the two lexems and generates a color mask for the target based on source.

func (*Lexer) Scan added in v0.2.0 

func (l *Lexer) Scan(text string) (xms []string)

Scan will break the text into lexemes and return them. A lexeme is either a string consisting of non-unicode.IsSpace characters, or a single newline character. If no lexemes found, nil is returned.

type LinedError 

type LinedError struct {
	Header string
	Line   int
	Err    error
}

LinedError appends line information to the error message. It is mainly used to test that errors are produced for correct lines.

func (*LinedError) Error 

func (e *LinedError) Error() string

func (*LinedError) Unwrap 

func (e *LinedError) Unwrap() error

type ProcessResult added in v0.3.0 

type ProcessResult struct {
	ExitCode int
	Stdout   string
	Stderr   string
}

ProcessResult contains the text printed to stdout and stderr by the process and the exit code returned upon termination.

type Processer 

type Processer interface {
	Run(io.Reader) (ProcessResult, error)
}

Processer interface abstracts away the concept of the executable under testing.

type ProcesserFunc 

type ProcesserFunc func(io.Reader) (ProcessResult, error)

ProcesserFunc represents an implementation of Processer that instead of a real OS-level process executes Go code.

func (ProcesserFunc) Run 

func (p ProcesserFunc) Run(r io.Reader) (ProcessResult, error)

Run will call the underlying Go function to compute the result.

type RichText added in v0.2.0 

type RichText struct {
	Str  string
	Mask []bool
}

RichText represents a text data with additional color metadata in a form of a bitmask. The characters may be either colored or uncolored. The _color_ might represent a literal color or a formatting style like bold or italics.

func (RichText) Colorful added in v0.2.0 

func (rt RichText) Colorful() bool

Colorful returns whether at least one character in Str has color.

func (RichText) Colorize added in v0.2.0 

func (rt RichText) Colorize(color aurora.Color) string

Colorize returns Str with ASCII escape codes actually embedded inside it to enable colors. The resulting string then can be printed on the screen and it'll be colorful, for example.

type SpyProcesser 

type SpyProcesser struct {
	Proc Processer
	// contains filtered or unexported fields
}

SpyProcesser is a test double that proxies another processer. It additionally stores the number of calls made to the Run function.

func (*SpyProcesser) CallCount 

func (p *SpyProcesser) CallCount() int

CallCount will return the number of times Run was called. Can be called concurrently.

func (*SpyProcesser) Run 

func (p *SpyProcesser) Run(r io.Reader) (ProcessResult, error)

Run will execute the Run function of the inner processer, but will also increase the call count by one.

type SpyStopwatcher 

type SpyStopwatcher struct {
	TLAtCall int
	// contains filtered or unexported fields
}

SpyStopwatcher implements Stopwatcher but instead of real time substitutes index sequence numbers. If time limit equals zero, then the time limit will never fire.

func (*SpyStopwatcher) Elapsed 

func (s *SpyStopwatcher) Elapsed() time.Duration

Elapsed will return the number of seconds that equals to the number of calls made to the TimeLimit method.

func (*SpyStopwatcher) TimeLimit 

func (s *SpyStopwatcher) TimeLimit() <-chan time.Duration

TimeLimit returns a channel that sends the TLAtCall number of seconds back at the TLAtCall-th call to the TimeLimit method.

type Stopwatcher 

type Stopwatcher interface {
	Elapsed() time.Duration
	TimeLimit() <-chan time.Duration
}

Stopwatcher abstracts away the concept of the stopwatch. At any time, one can look up the elapsed time. Additionally, one can be notified when the time is up.

type Test 

type Test struct {
	Input  string
	Output string
}

Test represents a single test case: an input and the expected output.

func ScanTest 

func ScanTest(testStr string) (Test, []error)

ScanTest parses a single test case: input and output, separated with the Input/Output separator. It also trims space around input and output line-wise. If separator is absent, it returns an error.

type TestEndCallbackFunc added in v0.2.0 

type TestEndCallbackFunc func(*TestingBatch, Test, int)

TestEndCallbackFunc represents a function to be called when a test case finishes execution. Usually, one would like to print the test case result information.

It accepts the pointer to the TestingBatch that contains verdict, time, program's error and output info. it also accepts the Test and the id of the Test.

type TestResult added in v0.2.0 

type TestResult struct {
	ID  int
	Err error
	Out ProcessResult
}

TestResult carries an output of the process given the ID-th test input and an error if any.

type TestStartCallbackFunc added in v0.2.0 

type TestStartCallbackFunc func(int)

TestStartCallbackFunc represents a function to be called before a test case will be launched. It accepts the id of the test case.

type TestingBatch 

type TestingBatch struct {
	Errs        map[int]error
	Outs        map[int]ProcessResult
	RichOuts    map[int][]RichText
	RichAnswers map[int][]RichText
	Lx          *Lexer

	Verdicts map[int]Verdict
	Times    map[int]time.Duration

	Proc   Processer
	Swatch Stopwatcher

	TestStartCallback TestStartCallbackFunc
	TestEndCallback   TestEndCallbackFunc
	// contains filtered or unexported fields
}

TestingBatch is responsible for running tests and evaluating the verdicts for tests. For each test case, the verdict and execution time are stored. It utilizer an instance of Processer to run tests, and an instance of Stopwatcher to track time limit. Optionally, user can set ResultPrinter to a custom function to output useful statistics about test case's result.

func NewTestingBatch 

func NewTestingBatch(inputs Inputs, proc Processer, swatch Stopwatcher) *TestingBatch

NewTestingBatch will initialize channels and maps inside TestingBatch and will assign respective dependency injections.

func (*TestingBatch) Run 

func (b *TestingBatch) Run()

Run will lauch test cases in parallel and then will wait for each test to finish or for the time to timelimit. When a test is finished the verdict and the time it took to execute are remembered. Additionally, ResultPrinter is called on the test case's statistics. When a time limit is reached, each not-yet-judged test is assigned TL verdict and the ResultPrinter is also called on each test.

type Verdict 

type Verdict int

Verdict represents a verdict asssigned by the judge. 

const (
	OK Verdict = iota
	IE
	WA
	RE
	TL
)

The set of all possible judge verdicts that can be assigned. The abbreviatons are due to competitive programming online judges. (Except for IE, that stands for Internal Error)

Source Files

View all Source files

Directories

Path Synopsis
cmd
cptest

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.