torus: Index | Files | Directories

package torus

import ""

Torus is a distributed storage system, allowing for the creation and coordination of sharded, replicated files. For more details, see the README at


Package Files

blockset.go config.go errors.go file.go file_cache.go heartbeat.go inode.go local_server.go metadata.go ring.go server.go storage.go version.go


const (
    CtxWriteLevel int = iota
const BlockRefByteSize = 8 * 3
const INodeRefByteSize = 8 * 2
const VolumeIDByteSize = 5
const (
    VolumeMax = 0x000000FFFFFFFFFF


var (
    // ErrBlockUnavailable is returned when a function fails to retrieve a known
    // block.
    ErrBlockUnavailable = errors.New("torus: block cannot be retrieved")

    // ErrINodeUnavailable is returned when a function fails to retrieve a known
    // INode.
    ErrINodeUnavailable = errors.New("torus: inode cannot be retrieved")

    // ErrBlockNotExist is returned when a function attempts to manipulate a
    // non-existent block.
    ErrBlockNotExist = errors.New("torus: block doesn't exist")

    // ErrClosed is returned when a function attempts to manipulate a Store
    // that is not currently open.
    ErrClosed = errors.New("torus: store is closed")

    // ErrInvalid is a locally invalid operation (such as Close()ing a nil file pointer)
    ErrInvalid = errors.New("torus: invalid operation")

    // ErrOutOfSpace is returned when the block storage is out of space.
    ErrOutOfSpace = errors.New("torus: out of space on block store")

    // ErrExists is returned if the entity already exists
    ErrExists = errors.New("torus: already exists")

    // ErrNotExist is returned if the entity doesn't already exist
    ErrNotExist = errors.New("torus: doesn't exist")

    // ErrAgain is returned if the operation was interrupted. The call was valid, and
    // may be tried again.
    ErrAgain = errors.New("torus: interrupted, try again")

    // ErrNoGlobalMetadata is returned if the metadata service hasn't been formatted.
    ErrNoGlobalMetadata = errors.New("torus: no global metadata available at mds")

    // ErrNonSequentialRing is returned if the ring's internal version number appears to jump.
    ErrNonSequentialRing = errors.New("torus: non-sequential ring")

    // ErrNoPeer is returned if the peer can't be found.
    ErrNoPeer = errors.New("torus: no such peer")

    // ErrCompareFailed is returned if the CAS operation failed to compare.
    ErrCompareFailed = errors.New("torus: compare failed")

    // ErrIsSymlink is returned if we're trying to modify a symlink incorrectly.
    ErrIsSymlink = errors.New("torus: is symlink")

    // ErrNotDir is returned if we're trying a directory operation on a non-directory path.
    ErrNotDir = errors.New("torus: not a directory")

    // ErrWrongVolumeType is returned if the operation cannot be performed on this type of volume.
    ErrWrongVolumeType = errors.New("torus: wrong volume type")

    // ErrNotSupported is returned if the interface doesn't implement the
    // requested subfunctionality.
    ErrNotSupported = errors.New("torus: not supported")

    // ErrLocked is returned if the resource is locked.
    ErrLocked = errors.New("torus: locked")

    // ErrLeaseNotFound is returned if the lease cannot be found.
    ErrLeaseNotFound = errors.New("torus: lease not found")

    // ErrUsage is returned if the command usage is wrong.
    ErrUsage = errors.New("torus: wrong command usage")
var BlockLog = capnslog.NewPackageLogger("", "blocklog")
var Version string

Version is set by build scripts, do not touch.

func InitMDS Uses

func InitMDS(name string, cfg Config, gmd GlobalMetadata, ringType RingType) error

InitMDS calls the specific init function provided by a metadata package.

func MarshalBlocksetToProto Uses

func MarshalBlocksetToProto(bs Blockset) ([]*models.BlockLayer, error)

func MkdirsFor Uses

func MkdirsFor(dir string) error

func RegisterBlockStore Uses

func RegisterBlockStore(name string, newFunc NewBlockStoreFunc)

func RegisterMetadataInit Uses

func RegisterMetadataInit(name string, newFunc InitMDSFunc)

RegisterMetadataInit is the hook used for implementations of MetadataServices to register their ways of creating base metadata to the system.

func RegisterMetadataService Uses

func RegisterMetadataService(name string, newFunc CreateMetadataServiceFunc)

RegisterMetadataService is the hook used for implementations of MetadataServices to register themselves to the system. This is usually called in the init() of the package that implements the MetadataService. A similar pattern is used in database/sql of the standard library.

func RegisterMetadataWipe Uses

func RegisterMetadataWipe(name string, newFunc WipeMDSFunc)

RegisterMetadataWipe is the hook used for implementations of MetadataServices to register their ways of deleting their metadata from the consistent store

func RegisterSetRing Uses

func RegisterSetRing(name string, newFunc SetRingFunc)

RegisterSetRing is the hook used for implementations of MetadataServices to register their ways of creating base metadata to the system.

func SetRing Uses

func SetRing(name string, cfg Config, r Ring) error

SetRing calls the specific SetRing function provided by a metadata package.

func WipeMDS Uses

func WipeMDS(name string, cfg Config) error

type BlockIterator Uses

type BlockIterator interface {
    Err() error
    Next() bool
    BlockRef() BlockRef
    Close() error

type BlockLayer Uses

type BlockLayer struct {
    Kind    BlockLayerKind
    Options string

type BlockLayerKind Uses

type BlockLayerKind int

type BlockLayerSpec Uses

type BlockLayerSpec []BlockLayer

type BlockRef Uses

type BlockRef struct {
    Index IndexID

BlockRef is the identifier for a unique block in the cluster.

func BlockFromProto Uses

func BlockFromProto(p *models.BlockRef) BlockRef

func BlockRefFromBytes Uses

func BlockRefFromBytes(b []byte) BlockRef

func ZeroBlock Uses

func ZeroBlock() BlockRef

func (BlockRef) BlockType Uses

func (b BlockRef) BlockType() BlockType

func (BlockRef) HasINode Uses

func (b BlockRef) HasINode(i INodeRef, t BlockType) bool

func (BlockRef) IsZero Uses

func (b BlockRef) IsZero() bool

func (*BlockRef) SetBlockType Uses

func (b *BlockRef) SetBlockType(t BlockType)

func (BlockRef) String Uses

func (b BlockRef) String() string

func (BlockRef) ToBytes Uses

func (b BlockRef) ToBytes() []byte

func (BlockRef) ToBytesBuf Uses

func (b BlockRef) ToBytesBuf(buf []byte)

func (BlockRef) ToProto Uses

func (b BlockRef) ToProto() *models.BlockRef

type BlockStore Uses

type BlockStore interface {
    HasBlock(ctx context.Context, b BlockRef) (bool, error)
    GetBlock(ctx context.Context, b BlockRef) ([]byte, error)
    WriteBlock(ctx context.Context, b BlockRef, data []byte) error
    WriteBuf(ctx context.Context, b BlockRef) ([]byte, error)
    DeleteBlock(ctx context.Context, b BlockRef) error
    NumBlocks() uint64
    UsedBlocks() uint64
    BlockIterator() BlockIterator
    BlockSize() uint64

BlockStore is the interface representing the standardized methods to interact with something storing blocks.

func CreateBlockStore Uses

func CreateBlockStore(kind string, name string, cfg Config, gmd GlobalMetadata) (BlockStore, error)

type BlockType Uses

type BlockType uint16
const (
    TypeBlock BlockType = iota

type Blockset Uses

type Blockset interface {
    // Length returns the number of blocks in the Blockset.
    Length() int
    // Kind returns the kind of the Blockset.
    Kind() uint32
    // GetBlock returns the ith block in the Blockset.
    GetBlock(ctx context.Context, i int) ([]byte, error)
    // PutBlock puts a block with data `b` into the Blockset as its ith block.
    // The block belongs to the given inode.
    PutBlock(ctx context.Context, inode INodeRef, i int, b []byte) error
    // GetLiveInodes returns the current INode representation of the Blockset.
    // The returned INode might not be synced.
    GetLiveINodes() *roaring.Bitmap
    // GetAllBlockRefs returns the BlockRef of the blocks in the Blockset.
    // The ith BlockRef in the returned slice is the Ref of the ith Block in the
    // Blockset.
    GetAllBlockRefs() []BlockRef

    // Marshal returns the bytes representation of the Blockset.
    Marshal() ([]byte, error)
    // Unmarshal parses the bytes representation of the Blockset and stores the result
    // in the Blockset.
    Unmarshal(data []byte) error
    // GetSubBlockset gets the sub-Blockset of the Blockset if exists.
    // If there is no sub-Blockset, nil will be returned.
    GetSubBlockset() Blockset
    // Truncate changes the length of the Blockset and the block. If the Blockset has less
    // blocks than the required size, truncate adds zero blocks. If the block has less bytes
    // than required size, truncate add bytes into block.
    Truncate(lastIndex int, blocksize uint64) error
    // Trim zeros the blocks in range [from, to).
    Trim(from, to int) error
    // String implements the fmt.Stringer interface.
    String() string

Blockset is the interface representing the standardized methods to interact with a set of blocks.

type Config Uses

type Config struct {
    DataDir         string
    StorageSize     uint64
    MetadataAddress string
    ReadCacheSize   uint64
    ReadLevel       ReadLevel
    WriteLevel      WriteLevel

    TLS *tls.Config

type CreateMetadataServiceFunc Uses

type CreateMetadataServiceFunc func(cfg Config) (MetadataService, error)

CreateMetadataServiceFunc is the signature of a constructor used to create a registered MetadataService.

type DebugMetadataService Uses

type DebugMetadataService interface {
    DumpMetadata(io.Writer) error

type File Uses

type File struct {
    ReadOnly bool
    // contains filtered or unexported fields

func (*File) Close Uses

func (f *File) Close() error

func (*File) Read Uses

func (f *File) Read(b []byte) (n int, err error)

func (*File) ReadAt Uses

func (f *File) ReadAt(b []byte, off int64) (n int, ferr error)

func (*File) Replaces Uses

func (f *File) Replaces() uint64

func (*File) Seek Uses

func (f *File) Seek(offset int64, whence int) (int64, error)

func (*File) Size Uses

func (f *File) Size() uint64

func (*File) SyncAllWrites Uses

func (f *File) SyncAllWrites() (INodeRef, error)

func (*File) SyncBlocks Uses

func (f *File) SyncBlocks() error

func (*File) SyncINode Uses

func (f *File) SyncINode(ctx context.Context) (INodeRef, error)

func (*File) Trim Uses

func (f *File) Trim(offset, length int64) error

Trim zeroes data in the middle of a file.

func (*File) Truncate Uses

func (f *File) Truncate(size int64) error

func (*File) Write Uses

func (f *File) Write(b []byte) (n int, err error)

func (*File) WriteAt Uses

func (f *File) WriteAt(b []byte, off int64) (n int, err error)

func (*File) WriteOpen Uses

func (f *File) WriteOpen() bool

type GlobalMetadata Uses

type GlobalMetadata struct {
    BlockSize        uint64
    DefaultBlockSpec BlockLayerSpec

type INodeID Uses

type INodeID uint64

INodeID represents a unique identifier for an INode.

type INodeIterator Uses

type INodeIterator struct {
    // contains filtered or unexported fields

func (*INodeIterator) Close Uses

func (i *INodeIterator) Close() error

func (*INodeIterator) Err Uses

func (i *INodeIterator) Err() error

func (*INodeIterator) INodeRef Uses

func (i *INodeIterator) INodeRef() INodeRef

func (*INodeIterator) Next Uses

func (i *INodeIterator) Next() bool

type INodeRef Uses

type INodeRef struct {
    INode INodeID
    // contains filtered or unexported fields

INodeRef is a reference to a unique INode in the cluster.

func INodeFromProto Uses

func INodeFromProto(p *models.INodeRef) INodeRef

func INodeRefFromBytes Uses

func INodeRefFromBytes(b []byte) INodeRef

func NewINodeRef Uses

func NewINodeRef(vol VolumeID, i INodeID) INodeRef

func ZeroINode Uses

func ZeroINode() INodeRef

func (INodeRef) Equals Uses

func (i INodeRef) Equals(x INodeRef) bool

func (INodeRef) String Uses

func (i INodeRef) String() string

func (INodeRef) ToBytes Uses

func (i INodeRef) ToBytes() []byte

func (INodeRef) ToProto Uses

func (i INodeRef) ToProto() *models.INodeRef

func (INodeRef) Volume Uses

func (i INodeRef) Volume() VolumeID

type INodeStore Uses

type INodeStore struct {
    // contains filtered or unexported fields

func NewINodeStore Uses

func NewINodeStore(bs BlockStore) *INodeStore

func (*INodeStore) Close Uses

func (b *INodeStore) Close() error

func (*INodeStore) DeleteINode Uses

func (b *INodeStore) DeleteINode(ctx context.Context, i INodeRef) error

func (*INodeStore) Flush Uses

func (b *INodeStore) Flush() error

func (*INodeStore) GetINode Uses

func (b *INodeStore) GetINode(ctx context.Context, i INodeRef) (*models.INode, error)

func (*INodeStore) INodeIterator Uses

func (b *INodeStore) INodeIterator() *INodeIterator

func (*INodeStore) WriteINode Uses

func (b *INodeStore) WriteINode(ctx context.Context, i INodeRef, inode *models.INode) error

type IndexID Uses

type IndexID uint64

IndexID represents a unique identifier for an Index.

type InitMDSFunc Uses

type InitMDSFunc func(cfg Config, gmd GlobalMetadata, ringType RingType) error

InitMDSFunc is the signature of a function which preformats a metadata service.

type MetadataKind Uses

type MetadataKind int
const (
    EtcdMetadata MetadataKind = iota

type MetadataService Uses

type MetadataService interface {
    GetVolumes() ([]*models.Volume, VolumeID, error)
    GetVolume(volume string) (*models.Volume, error)
    NewVolumeID() (VolumeID, error)
    Kind() MetadataKind

    // GlobalMetadata backing struct must be instantiated upon the
    // service creation. If it can not be created the MetadataService
    // creation must fail.
    GlobalMetadata() GlobalMetadata

    // Returns a UUID based on the underlying datadir. Should be
    // unique for every created datadir.
    UUID() string

    GetRing() (Ring, error)
    SubscribeNewRings(chan Ring)
    UnsubscribeNewRings(chan Ring)
    SetRing(ring Ring) error

    WithContext(ctx context.Context) MetadataService

    GetLease() (int64, error)
    RenewLease(int64) error
    RegisterPeer(lease int64, pi *models.PeerInfo) error
    GetPeers() (PeerInfoList, error)

    Close() error

    CommitINodeIndex(VolumeID) (INodeID, error)
    GetINodeIndex(VolumeID) (INodeID, error)
    GetLockStatus(vid uint64) string

MetadataService is the interface representing the basic ways to manipulate consistently stored fileystem metadata.

func CreateMetadataService Uses

func CreateMetadataService(name string, cfg Config) (MetadataService, error)

CreateMetadataService calls the constructor of the specified MetadataService with the provided address.

type ModifyableRing Uses

type ModifyableRing interface {
    ChangeReplication(r int) (Ring, error)

type NewBlockStoreFunc Uses

type NewBlockStoreFunc func(string, Config, GlobalMetadata) (BlockStore, error)

type PeerInfoList Uses

type PeerInfoList []*models.PeerInfo

func (PeerInfoList) AndNot Uses

func (pi PeerInfoList) AndNot(b PeerList) PeerInfoList

func (PeerInfoList) GetWeights Uses

func (pi PeerInfoList) GetWeights() map[string]int

func (PeerInfoList) HasUUID Uses

func (pi PeerInfoList) HasUUID(uuid string) bool

func (PeerInfoList) Intersect Uses

func (pi PeerInfoList) Intersect(b PeerInfoList) PeerInfoList

func (PeerInfoList) PeerList Uses

func (pi PeerInfoList) PeerList() PeerList

func (PeerInfoList) UUIDAt Uses

func (pi PeerInfoList) UUIDAt(uuid string) int

func (PeerInfoList) Union Uses

func (pi PeerInfoList) Union(b PeerInfoList) PeerInfoList

type PeerList Uses

type PeerList []string

func (PeerList) AndNot Uses

func (pl PeerList) AndNot(b PeerList) PeerList

func (PeerList) Has Uses

func (pl PeerList) Has(uuid string) bool

func (PeerList) IndexAt Uses

func (pl PeerList) IndexAt(uuid string) int

func (PeerList) Intersect Uses

func (pl PeerList) Intersect(b PeerList) PeerList

func (PeerList) Union Uses

func (pl PeerList) Union(b PeerList) PeerList

type PeerPermutation Uses

type PeerPermutation struct {
    Replication int
    Peers       PeerList

type ReadLevel Uses

type ReadLevel int
const (
    ReadBlock ReadLevel = iota

func ParseReadLevel Uses

func ParseReadLevel(s string) (rl ReadLevel, err error)

type Ring Uses

type Ring interface {
    GetPeers(key BlockRef) (PeerPermutation, error)
    Members() PeerList

    Describe() string
    Type() RingType
    Version() int

    Marshal() ([]byte, error)

type RingAdder Uses

type RingAdder interface {
    AddPeers(PeerInfoList) (Ring, error)

type RingRemover Uses

type RingRemover interface {
    RemovePeers(PeerList) (Ring, error)

type RingType Uses

type RingType int

type Server Uses

type Server struct {
    Blocks BlockStore
    MDS    MetadataService
    INodes *INodeStore

    Cfg Config

    ReplicationOpen bool
    // contains filtered or unexported fields

Server is the type representing the generic distributed block store.

func NewMemoryServer Uses

func NewMemoryServer() *Server

func NewServer Uses

func NewServer(cfg Config, metadataServiceKind, blockStoreKind string) (*Server, error)

func NewServerByImpl Uses

func NewServerByImpl(cfg Config, mds MetadataService, blocks BlockStore) (*Server, error)

func (*Server) AddTimeoutCallback Uses

func (s *Server) AddTimeoutCallback(f func(uuid string))

func (*Server) BeginHeartbeat Uses

func (s *Server) BeginHeartbeat(addr *url.URL) error

BeginHeartbeat spawns a goroutine for heartbeats. Non-blocking.

func (*Server) Close Uses

func (s *Server) Close() error

func (*Server) CreateFile Uses

func (s *Server) CreateFile(volume *models.Volume, inode *models.INode, blocks Blockset) (*File, error)

func (*Server) Debug Uses

func (s *Server) Debug(w io.Writer) error

Debug writes a bunch of debug output to the io.Writer.

func (*Server) ExtendContext Uses

func (s *Server) ExtendContext(ctx context.Context) context.Context

func (*Server) GetPeerMap Uses

func (s *Server) GetPeerMap() map[string]*models.PeerInfo

func (*Server) Lease Uses

func (s *Server) Lease() int64

func (*Server) UpdatePeerMap Uses

func (s *Server) UpdatePeerMap() map[string]*models.PeerInfo

func (*Server) UpdateRebalanceInfo Uses

func (s *Server) UpdateRebalanceInfo(ri *models.RebalanceInfo)

type SetRingFunc Uses

type SetRingFunc func(cfg Config, r Ring) error

type Store Uses

type Store interface {
    Kind() string
    Flush() error
    Close() error

Store is the interface that represents methods that should be common across all types of storage providers.

type VolumeID Uses

type VolumeID uint64

VolumeID represents a unique identifier for a Volume.

func (VolumeID) ToBytes Uses

func (v VolumeID) ToBytes() []byte

type WipeMDSFunc Uses

type WipeMDSFunc func(cfg Config) error

type WriteLevel Uses

type WriteLevel int
const (
    WriteAll WriteLevel = iota

func ParseWriteLevel Uses

func ParseWriteLevel(s string) (wl WriteLevel, err error)


blockblock provides the implementation of the "block" volume type, using a Torus file as a block device.
block/aoeaoe provides the implementation of an ATA over Ethernet server, backed by a Torus block volume.
blocksetblockset provides a registry of BlockLayers, that can be (Un)Marshaled and retrieve blocks from a Torus storage interface.
distributordistributor is a complex implementation of a Torus storage interface, that understands rebalancing it's underlying storage and fetching data from peers, as necessary.
distributor/protocolsprotocols is the metapackage for the RPC protocols for how Torus' storage layer communicates with other storage servers.
distributor/rebalancerebalance provides the implementation of the rebalancer, which continually checks the data stored on a host, knows where data should live, and moves it to the appropriate servers.
gcgc provides the Torus interface for how garbage collection is implemented.
internal/flagconfigflagconfig is a generic set of flags dedicated to configuring a Torus client.
internal/nbdPackage nbd uses the Linux NBD layer to emulate a block device in user space
metadatametadata is the metapackage for the implementations of the metadata interface, for each potential backend.
modelsmodels is the package containing all the protos used for serializing data for Torus.
ringring is the package containing implementations of the consistent hash ring, a pure function which provides a permutation of peers where a block can live, known by all members of the cluster.
storagestorage is the package which implements the underlying, on-disk storage API for Torus servers.

Package torus imports 18 packages (graph) and is imported by 68 packages. Updated 2017-02-22. Refresh now. Tools for package owners.