Documentation
¶
Index ¶
- func RunTest(name string, clusterCfg cluster.Config, testCfg TestConfig) error
- type TestCase
- func (t *TestCase) TestConcurrentRead() error
- func (tc *TestCase) TestCorruptAndUnblock() error
- func (tc *TestCase) TestCorruptLoseTract() error
- func (tc *TestCase) TestCorruptTract() error
- func (tc *TestCase) TestCuratorConsistency() error
- func (tc *TestCase) TestCuratorPartition() error
- func (tc *TestCase) TestDrain() error
- func (tc *TestCase) TestGCAfterInterruptedRerepl() error
- func (tc *TestCase) TestGCDeletedTract() error
- func (tc *TestCase) TestGCStaleTract() error
- func (tc *TestCase) TestInterruptedRerepl() error
- func (tc *TestCase) TestInterruptedRereplFixversion() error
- func (tc *TestCase) TestInterruptedRereplRead() error
- func (tc *TestCase) TestMasterConsistency() error
- func (tc *TestCase) TestMasterPartition() error
- func (tc *TestCase) TestMissingTract() error
- func (tc *TestCase) TestOnlyOneHost() error
- func (tc *TestCase) TestPack() error
- func (tc *TestCase) TestPackFailure() error
- func (tc *TestCase) TestRSEncode() error
- func (tc *TestCase) TestRSRecovery() error
- func (tc *TestCase) TestReadEOF() error
- func (tc *TestCase) TestReadOnly() error
- func (tc *TestCase) TestRerepl() error
- func (tc *TestCase) TestRereplReadOnly() error
- func (tc *TestCase) TestStorageHint() error
- func (tc *TestCase) TestStorageMigration() error
- func (tc *TestCase) TestTimes() error
- func (tc *TestCase) TestWriteReadAcrossTracts() error
- func (tc *TestCase) TestWriteReadLong() error
- func (tc *TestCase) TestWriteReadNoFailure() error
- func (tc *TestCase) TestWriteReadRaftFailure() error
- func (tc *TestCase) TestWriteReadWithFailureLong() error
- type TestConfig
- type TestResult
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type TestCase ¶
type TestCase struct {
// contains filtered or unexported fields
}
TestCase includes everyting that is needed to run a test of testblb.
func (*TestCase) TestConcurrentRead ¶
TestConcurrentRead tests concurrently reading from disjoint intervals in the same blob.
func (*TestCase) TestCorruptAndUnblock ¶
TestCorruptAndUnblock trashes all replicas of a tract except one and expects to be able to write to it.
func (*TestCase) TestCorruptLoseTract ¶
TestCorruptLoseTract tests that a tract can be successfully rereplicated in the case of mixed data corruption and server lost failure, as long as there is at least one replica available.
func (*TestCase) TestCorruptTract ¶
TestCorruptTract corrupts a tract on a tract server that owns it. The tract server should detect that and finally the curator should re-replicate that tract.
- Create a blob and write some data to it.
- Pick a tract server that owns the first tract and corrupt it.
- Wait until the tract server detects it and the tract gets re-replicated.
func (*TestCase) TestCuratorConsistency ¶
TestCuratorConsistency tests the consistency check functionality of the curator. First it checks that it can pass a check. Then it corrupts the state of one curator and checks that it causes a check failure.
func (*TestCase) TestCuratorPartition ¶
TestCuratorPartition tests that during a network partition that a quorum of curators can't talk to each other we'll lose the availability of the curator because we opt for consistency instead of availability during network partitions. And when the partition gets healed(partially healed) that a quorum of curators of a curator group can talk the system will be available again eventually.
- Create a blob and it should succeed.
- Partition curator replicas of each curator gorup that no quorum can talk to each other.
- Eventually the leaders of each curator group will step down.
- Create will fail.
- Heal the partition that a quorum of each curator group can talk with each other, a new leader of each group will be elected.
- Create should succeed again.
func (*TestCase) TestDrain ¶
TestDrain tests that we can set control flags on a disk on a tractserver and have it drain (curator replicates tracts off of it).
func (*TestCase) TestGCAfterInterruptedRerepl ¶
TestGCAfterInterruptedRerepl exercises gc-ing a up-to-date tract on a tractserver not in the repl group due to interrupted rerepl.
func (*TestCase) TestGCDeletedTract ¶
TestGCDeletedTract exercises gc-ing tracts in a deleted blob.
func (*TestCase) TestGCStaleTract ¶
TestGCStaleTract exercises gc-ing a stale tract.
func (*TestCase) TestInterruptedRerepl ¶
TestInterruptedRerepl exercises the situation that might result if a curator crashed in the middle of a re-replication, after it had bumped the version on some tractservers, but maybe not all, and its durable version number has not been updated.
func (*TestCase) TestInterruptedRereplFixversion ¶
TestInterruptedRereplFixversion is similar to TestInterruptedRerepl with the distinction that in a replication group of 3, we crash one host, bump the version on the second (simulating the curator crashed in the middle of re-replication), and only leave the third intact. Writting to the tract should eventually success after all background rereplication and version fixing.
func (*TestCase) TestInterruptedRereplRead ¶
TestInterruptedRereplRead exercises the situation that might result if a curator crashed in the middle of a re-replication, after it had bumped the version on all tractservers but not updated its durable state yet.
func (*TestCase) TestMasterConsistency ¶
TestMasterConsistency tests the consistency check functionality of the master. First it checks that it can pass a check. Then it corrupts the state of one master and checks that it causes a check failure.
func (*TestCase) TestMasterPartition ¶
TestMasterPartition tests that during a network partition that a quorum of masters can't talk to each other we'll lose the availability of the master because we opt for consistency instead of availability during network partitions. And when the partition gets healed(partially healed) that a quorum of masters can talk the system will be available again eventually.
- Create a blob and it should succeed.
- Partition master replicas that no quorum can talk to each other.
- Eventually the leader of the master group will step down.
- Create will fail.
- Heal the partition that a quorum can talk with each other, a new leader will be elected.
- Create should succeed again.
func (*TestCase) TestMissingTract ¶
TestMissingTract tests that a tract exists in curator's state but missing on one of tract servers. This should be detected by curator and the tract will be re-replicated.
func (*TestCase) TestOnlyOneHost ¶
TestOnlyOneHost tests that a tract can be written to (eventually) even if every replica except one is down.
func (*TestCase) TestPackFailure ¶
TestPackFailure tests that PackTracts fails if sources are corrupted.
func (*TestCase) TestRSEncode ¶
TestRSEncode tests that a tractserver can RSEncode from/to other tractservers.
func (*TestCase) TestRSRecovery ¶
TestRSRecovery tests that the curator can recover RS data from various types of failures. We test a bunch of things serially to amoritize the setup time.
func (*TestCase) TestReadEOF ¶
TestReadEOF tests whether client can get EOF error after reading a whole blob.
func (*TestCase) TestReadOnly ¶
TestReadOnly tests the read-only mode of the curator.
func (*TestCase) TestRerepl ¶
TestRerepl exercises a single rereplicate operation.
func (*TestCase) TestRereplReadOnly ¶
TestRereplReadOnly verifies read-only tract can be properly handled for writes. Writing to a read-only tract should fail and trigger rereplication. After that, further write should succeed.
- Create a blob and write some data to it.
- Pick a replica of the first tract and make it read-only.
- Wait until the tract gets rereplicated and verify with a write.
func (*TestCase) TestStorageHint ¶
TestStorageHint tests functionality related to storage hints and classes.
func (*TestCase) TestStorageMigration ¶
TestStorageMigration tests that the curator can erasure-code data.
func (*TestCase) TestWriteReadAcrossTracts ¶
TestWriteReadAcrossTracts tests writing to different parts of an existing blob and the write might span multiple tracts. After the write the blob should be in an expected state.
func (*TestCase) TestWriteReadLong ¶
TestWriteReadLong writes data to blb, reads it out and verifies we get what we wrote.
func (*TestCase) TestWriteReadNoFailure ¶
TestWriteReadNoFailure tests that write and read operation should work as expected in case that there's no failures.
func (*TestCase) TestWriteReadRaftFailure ¶
TestWriteReadRaftFailure tests that replicas of master and curator can fail arbitrarily but as long as a quorum of master or curator are up the whole Blb cluster should still function.
- Create blob and write some data to it.
- Kill all master and curator replicas and restart a quorum of instance.
- Read back the original blob so we know the blob metadata is still accessible.
- Create, write and verify a new Blob so we know masters & curators are still writable.
func (*TestCase) TestWriteReadWithFailureLong ¶
TestWriteReadWithFailureLong is similar to WriteAndRead, with the distinction that some master/curator/tractserver instances are killed and restarted randomly from time to time.
type TestConfig ¶
type TestConfig struct {
TestTime time.Duration // How long we run the test.
WriteInterval time.Duration // How often we write.
FailureInterval time.Duration // How often we kill a server.
WriteSizeLowMB int // The smallest write size in mb.
WriteSizeHighMB int // The largest write size in mb.
MaxReplFactor int // Maximum replication factor.
NumWriters int // Number of concurrent writers.
DisableCache bool // Whether client cache is disabled.
TestPattern string // Regex matching tests to run.
TestLong bool // Whether tests ended with "Long" should be run (they tend to last for an extended period of time and are disabled by default).
NumWorkers int // Number of test workers to run in parallel.
}
TestConfig includes configuration parameters for testing.
type TestResult ¶
func RunController ¶
func RunController(testCfg TestConfig, binDir, workDir string, neededPorts int) (ret []TestResult)
RunController is called by the controller process to initialize resources and spawn worker processes to run tests that need to be run. The test results will be returned as a map which has test names as keys and results as values.
Source Files
¶
- case.go
- config.go
- controller.go
- partition_failures.go
- test_concurrent_read.go
- test_corrupt.go
- test_corrupt_and_unblock.go
- test_corrupt_lose_tract.go
- test_curator_consistency.go
- test_curator_partition.go
- test_drain.go
- test_gc_after_interrupted_rerepl.go
- test_gc_deleted_tract.go
- test_gc_stale_tract.go
- test_interrupted_rerepl.go
- test_interrupted_rerepl_fixversion.go
- test_interrupted_rerepl_read.go
- test_master_consistency.go
- test_master_partition.go
- test_missing_tract.go
- test_only_one_host.go
- test_pack.go
- test_pack_failure.go
- test_read_eof.go
- test_readonly.go
- test_rerepl.go
- test_rerepl_read_only.go
- test_rs_recovery.go
- test_rsencode.go
- test_storage_hint.go
- test_storage_migration.go
- test_times.go
- test_write_read_across_tracts.go
- test_write_read_long.go
- test_write_read_no_failure.go
- test_write_read_raft_failure.go
- test_write_read_with_failure_long.go