storageclient

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 1, 2020 License: Apache-2.0 Imports: 42 Imported by: 2

Documentation

Index

Constants

View Source 
const (
	PersistDirectory            = "storageclient"
	PersistFilename             = "storageclient.json"
	PersistStorageClientVersion = "1.0"
	DxPathRoot                  = "dxfiles"
)

Files and directories related constant

View Source 
const (
	DefaultMaxDownloadSpeed = 0
	DefaultMaxUploadSpeed   = 0
	DefaultPacketSize       = 4 * 4096

	// frequency to check whether storage client is online
	OnlineCheckFrequency = time.Second * 10

	// the amount of time that can pass for processing activating worker pool
	WorkerActivateTimeout = time.Minute * 5

	// how long to wait for a worker after a worker failed to perform a download task.
	DownloadFailureCooldown = time.Second * 3

	// how many times a bad host's timeout/cool down can be doubled before a maximum cool down is reached.
	MaxConsecutivePenalty = 10
)

StorageClient Settings, where 0 means unlimited

View Source 
const (
	// DefaultMaxMemory available
	DefaultMaxMemory = uint64(3 * 1 << 28)
)

Variables

View Source 
var (
	// healthCheckInterval defines the maximum amount of time that should pass
	// in between checking the health of a file or directory.
	HealthCheckInterval = 30 * time.Minute

	// MaxConsecutiveSegmentUploads is the maximum number of segment before rebuilding the heap.
	MaxConsecutiveSegmentUploads = 100

	// repairStuckChunkInterval defines how long the storage client sleeps between
	// trying to repair a stuck chunk. The uploadHeap prioritizes stuck chunks
	// so this interval is to allow time for unstuck chunks to be repaired.
	// Ideally the uploadHeap is spending 95% of its time repairing unstuck
	// chunks.
	RepairStuckSegmentInterval = 10 * time.Minute

	// uploadAndRepairErrorSleepDuration indicates how long a upload process
	// should sleep before retrying if there is an error fetching the metadata
	// of the root directory of the storage client's filesystem.
	UploadAndRepairErrorSleepDuration = 15 * time.Minute

	// RemoteRepairDownloadThreshold indicates the threshold in percent under
	// which the storage client starts repairing a file that is not available on disk
	RemoteRepairDownloadThreshold = 0.125

	// UploadFailureCoolDown is the initial time of punishment while upload consecutive fails
	// the punishment time shows exponential growth
	UploadFailureCoolDown = 3 * time.Second
)

Default params about upload/download process

View Source 
var (
	// ErrNoContractsWithHost will be used when the client has no contract with host
	// the worker will be terminated
	ErrNoContractsWithHost = errors.New("no contract with host which is need to terminate")

	// ErrUnableRetrieveHostInfo is used when host information cannot be retrieved
	// worker will be terminated
	ErrUnableRetrieveHostInfo = errors.New("can't retrieve host info")

	// ErrContractRenewing is used when client and host is renewing contract
	// the worker will return directly
	ErrContractRenewing = errors.New("client and host is renewing contract")
)

Functions

func CalculateProofRanges 

func CalculateProofRanges(actions []storage.UploadAction, oldNumSectors uint64) []merkle.SubTreeLimit

CalculateProofRanges will calculate the proof ranges which is used to verify a pre-modification Merkle diff proof for the specified actions.

func ModifyLeaves 

func ModifyLeaves(leafHashes []common.Hash, actions []storage.UploadAction, numSectors uint64) []common.Hash

ModifyLeaves will modify the leaf hashes of a Merkle diff proof to verify a post-modification Merkle diff proof for the specified actions.

func ModifyProofRanges 

func ModifyProofRanges(proofRanges []merkle.SubTreeLimit, actions []storage.UploadAction, numSectors uint64) []merkle.SubTreeLimit

ModifyProofRanges will modify the proof ranges produced by calculateProofRanges to verify a post-modification Merkle diff proof for the specified actions.

func NewRevision 

func NewRevision(current types.StorageContractRevision, cost *big.Int) types.StorageContractRevision

NewRevision update current storage contract revision with its revision number incremented, and cost transferred from the client to the host.

Types

type ActiveContractsAPI 

type ActiveContractsAPI struct {
	ID           storage.ContractID
	HostID       enode.ID
	AbleToUpload bool
	AbleToRenew  bool
	Canceled     bool
}

ActiveContractsAPI is used to re-format the contract information that is going to be displayed on the console

type ActiveContractsAPIDisplay 

type ActiveContractsAPIDisplay struct {
	ContractID   string
	HostID       string
	AbleToUpload bool
	AbleToRenew  bool
	Canceled     bool
}

ActiveContractsAPIDisplay is used to re-format the contract information that is going to be displayed on the console

type ContractMetaDataAPIDisplay 

type ContractMetaDataAPIDisplay struct {
	ID                     string
	EnodeID                enode.ID
	LatestContractRevision types.StorageContractRevision
	StartHeight            string
	EndHeight              string

	ContractBalance string

	UploadCost   string
	DownloadCost string
	StorageCost  string
	TotalCost    string
	GasCost      string
	ContractFee  string

	UploadAbility string
	RenewAbility  string
	Canceled      string
}

ContractMetaDataAPIDisplay is the data structure used for console contract information display purposes

type PrivateStorageClientAPI 

type PrivateStorageClientAPI struct {
	// contains filtered or unexported fields
}

PrivateStorageClientAPI defines the object used to call eligible APIs that are used to configure settings

func NewPrivateStorageClientAPI 

func NewPrivateStorageClientAPI(sc *StorageClient) *PrivateStorageClientAPI

NewPrivateStorageClientAPI initialize PrivateStorageClientAPI object which implemented a bunch of API methods

func (*PrivateStorageClientAPI) PeriodCost 

func (api *PrivateStorageClientAPI) PeriodCost() storage.PeriodCost

PeriodCost will get the client's period cost which specifies cost that storage client needs to pay within one period cycle. It includes cost for all contracts

func (*PrivateStorageClientAPI) SetConfig 

func (api *PrivateStorageClientAPI) SetConfig(settings map[string]string) (resp string, err error)

SetConfig will configure the client setting based on the user input data

func (*PrivateStorageClientAPI) SetPaymentAddress 

func (api *PrivateStorageClientAPI) SetPaymentAddress(addrStr string) bool

SetPaymentAddress configure the account address used to sign the storage contract, which has and can only be the address of the local wallet.

type PublicStorageClientAPI 

type PublicStorageClientAPI struct {
	// contains filtered or unexported fields
}

PublicStorageClientAPI defines the object used to call eligible public APIs are used to acquire information

func NewPublicStorageClientAPI 

func NewPublicStorageClientAPI(sc *StorageClient) *PublicStorageClientAPI

NewPublicStorageClientAPI initialize PublicStorageClientAPI object which implemented a bunch of API methods

func (*PublicStorageClientAPI) Config 

func (api *PublicStorageClientAPI) Config() (setting storage.ClientSettingAPIDisplay)

Config will retrieve the current storage client settings

func (*PublicStorageClientAPI) Contract 

func (api *PublicStorageClientAPI) Contract(contractID string) (detail ContractMetaDataAPIDisplay, err error)

Contract will retrieve detailed contract information

func (*PublicStorageClientAPI) Contracts 

func (api *PublicStorageClientAPI) Contracts() (activeContracts []ActiveContractsAPIDisplay)

Contracts will retrieve all active contracts and display their general information

func (*PublicStorageClientAPI) DownloadSync 

func (api *PublicStorageClientAPI) DownloadSync(remoteFilePath, localPath string) (string, error)

DownloadSync is used to download remote file by sync mode NOTE: RPC not support async download, because it is stateless, should block until download task done.

func (*PublicStorageClientAPI) GetRenewWindow 

func (api *PublicStorageClientAPI) GetRenewWindow() string

GetRenewWindow return the renew window value

func (*PublicStorageClientAPI) Host 

func (api *PublicStorageClientAPI) Host(id string) (host storage.HostInfo, err error)

Host will retrieve a specific storage host information from the storage host manager based on the host id

func (*PublicStorageClientAPI) HostRank 

func (api *PublicStorageClientAPI) HostRank() (evaluation []storagehostmanager.StorageHostRank)

HostRank will retrieve the rankings of the storage hosts. The ranking information also includes detailed evaluation break down

func (*PublicStorageClientAPI) Hosts 

func (api *PublicStorageClientAPI) Hosts() (hosts []storage.HostInfo)

Hosts will retrieve the current storage hosts from the storage host manager

func (*PublicStorageClientAPI) PaymentAddress 

func (api *PublicStorageClientAPI) PaymentAddress() (common.Address, error)

PaymentAddress get the account address used to sign the storage contract. If not configured, the first address in the local wallet will be used as the paymentAddress by default.

func (*PublicStorageClientAPI) Upload 

func (api *PublicStorageClientAPI) Upload(source string, dxPath string) (string, error)

Upload their local files to hosts made contract with

type StorageClient 

type StorageClient struct {

	//storage client is used as the address to sign the storage contract and pays for the money
	PaymentAddress common.Address
	// contains filtered or unexported fields
}

StorageClient contains fields that are used to perform StorageHost selection operation, file uploading, downloading operations, and etc.

func New 

func New(persistDir string) (*StorageClient, error)

New initializes StorageClient object

func (*StorageClient) AccountManager 

func (client *StorageClient) AccountManager() *accounts.Manager

AccountManager will be used to acquire the account manager object which will be used to sign the contract, find the account address, and etc.

func (*StorageClient) ActiveContracts 

func (client *StorageClient) ActiveContracts() (activeContracts []ActiveContractsAPIDisplay)

ActiveContracts will retrieve all active contracts, reformat them, and return them back

func (*StorageClient) Append 

func (client *StorageClient) Append(sp storage.Peer, data []byte, hostInfo *storage.HostInfo) (common.Hash, error)

Append will send the given data to host and return the merkle root of data

func (*StorageClient) ChainConfig 

func (client *StorageClient) ChainConfig() *params.ChainConfig

ChainConfig will be used to retrieve the current chain configuration

func (*StorageClient) CheckAndUpdateConnection 

func (client *StorageClient) CheckAndUpdateConnection(peerNode *enode.Node)

CheckAndUpdateConnection will check the connection between client and host. If there are no contracts signed between the two, the connection will be updated from the static connection to dynamic connection

func (*StorageClient) Close 

func (client *StorageClient) Close() error

Close method will be used to send storage

func (*StorageClient) ContractDetail 

func (client *StorageClient) ContractDetail(contractID storage.ContractID) (detail storage.ContractMetaData, exists bool)

ContractDetail will return the detailed contract information

func (*StorageClient) CurrentBlock 

func (client *StorageClient) CurrentBlock() *types.Block

CurrentBlock is used to retrieve the current block number

func (*StorageClient) DeleteFile 

func (client *StorageClient) DeleteFile(path storage.DxPath) error

DeleteFile will delete from the file system file set. The file wil also be deleted from the disk

func (*StorageClient) DirInfo 

func (client *StorageClient) DirInfo(dxPath storage.DxPath) (storage.DirectoryInfo, error)

DirInfo returns the Directory Information of the dxdir

func (*StorageClient) DirList 

func (client *StorageClient) DirList(dxPath storage.DxPath) ([]storage.DirectoryInfo, []storage.UploadFileInfo, error)

DirList get directories and files in the dxdir

func (*StorageClient) Download 

func (client *StorageClient) Download(sp storage.Peer, root common.Hash, offset, length uint32, hostInfo *storage.HostInfo) ([]byte, error)

Download requests for a single section and returns the requested data. A Merkle proof is always requested.

func (*StorageClient) DownloadAsync 

func (client *StorageClient) DownloadAsync(p storage.DownloadParameters) error

DownloadAsync will perform a file download without blocking until the download is finished

func (*StorageClient) DownloadSync 

func (client *StorageClient) DownloadSync(p storage.DownloadParameters) error

DownloadSync performs a file download and blocks until the download is finished.

func (*StorageClient) GetFileSystem 

func (client *StorageClient) GetFileSystem() filesystem.FileSystem

GetFileSystem will get the file system

func (*StorageClient) GetHostAnnouncementWithBlockHash 

func (client *StorageClient) GetHostAnnouncementWithBlockHash(blockHash common.Hash) (hostAnnouncements []types.HostAnnouncement, number uint64, errGet error)

GetHostAnnouncementWithBlockHash will get the HostAnnouncements and block height through the hash of the block

func (*StorageClient) GetPaymentAddress 

func (client *StorageClient) GetPaymentAddress() (common.Address, error)

GetPaymentAddress get the account address used to sign the storage contract. If not configured, the first address in the local wallet will be used as the paymentAddress by default.

func (*StorageClient) GetPoolNonce 

func (client *StorageClient) GetPoolNonce(ctx context.Context, addr common.Address) (uint64, error)

GetPoolNonce returns the canonical nonce for the managed or un-managed account

func (*StorageClient) GetStorageHostManager 

func (client *StorageClient) GetStorageHostManager() *storagehostmanager.StorageHostManager

GetStorageHostManager will be used to acquire the storage host manager

func (*StorageClient) GetStorageHostSetting 

func (client *StorageClient) GetStorageHostSetting(hostEnodeID enode.ID, hostEnodeURL string, config *storage.HostExtConfig) error

GetStorageHostSetting will be used to get the storage host's external setting based on the peerID provided

func (*StorageClient) GetTxByBlockHash 

func (client *StorageClient) GetTxByBlockHash(blockHash common.Hash) (types.Transactions, error)

GetTxByBlockHash will be used to get the detailed transaction by using the block hash

func (*StorageClient) IsContractSignedWithHost 

func (client *StorageClient) IsContractSignedWithHost(hostNode *enode.Node) bool

IsContractSignedWithHost is used to check if the client has signed any contract with the storage host provided by the user

func (*StorageClient) Online 

func (client *StorageClient) Online() bool

Online will be used to indicate if the local node is connected to the internet

func (*StorageClient) Read 

func (client *StorageClient) Read(sp storage.Peer, w io.Writer, req storage.DownloadRequest, cancel <-chan struct{}, hostInfo *storage.HostInfo) (err error)

Download calls the Read RPC, writing the requested data to w NOTE: The RPC can be cancelled (with a granularity of one section) via the cancel channel.

func (*StorageClient) RetrieveClientSetting 

func (client *StorageClient) RetrieveClientSetting() (setting storage.ClientSetting)

RetrieveClientSetting will return the current storage client setting

func (*StorageClient) RevisionOrRenewingDone 

func (client *StorageClient) RevisionOrRenewingDone(hostID enode.ID)

RevisionOrRenewingDone indicates that the contract finished renewing

func (*StorageClient) SelfEnodeURL 

func (client *StorageClient) SelfEnodeURL() string

SelfEnodeURL retrieves the local node's enodeURL, used to avoid storing self information inf the storage host manager

func (*StorageClient) SendStorageContractCreateTx 

func (client *StorageClient) SendStorageContractCreateTx(clientAddr common.Address, input []byte) (common.Hash, error)

SendStorageContractCreateTx is used to send the contract create transaction to the transaction pool

func (*StorageClient) SendTx 

func (client *StorageClient) SendTx(ctx context.Context, signedTx *types.Transaction) error

SendTx will be used to send the transaction to the transaction pool

func (*StorageClient) SetClientSetting 

func (client *StorageClient) SetClientSetting(setting storage.ClientSetting) (err error)

SetClientSetting will config the client setting based on the value provided it will set the bandwidth limit, rentPayment, and ipViolation check By setting the rentPayment, the contract maintenance

func (*StorageClient) SetupConnection 

func (client *StorageClient) SetupConnection(enodeURL string) (storage.Peer, error)

SetupConnection will establish the secure P2P connection with the node provided

func (*StorageClient) Start 

func (client *StorageClient) Start(b storage.EthBackend, apiBackend ethapi.Backend) (err error)

Start controls go routine checking and updating process

func (*StorageClient) SubscribeChainChangeEvent 

func (client *StorageClient) SubscribeChainChangeEvent(ch chan<- core.ChainChangeEvent) event.Subscription

SubscribeChainChangeEvent will be used to get block information every time a change happened in the blockchain

func (*StorageClient) SuggestPrice 

func (client *StorageClient) SuggestPrice(ctx context.Context) (*big.Int, error)

SuggestPrice returns the recommended gas price

func (*StorageClient) Syncing 

func (client *StorageClient) Syncing() bool

Syncing will be used to indicate if the local node is syncing with the blockchain

func (*StorageClient) TryToRenewOrRevise 

func (client *StorageClient) TryToRenewOrRevise(hostID enode.ID) bool

TryToRenewOrRevise will be used to check if the contract is currently in the middle of the revision

func (*StorageClient) Upload 

func (client *StorageClient) Upload(up storage.FileUploadParams) error

Upload instructs the storage client to start tracking a file. The storage client will automatically upload and repair tracked files using a background loop.

func (*StorageClient) Write 

func (client *StorageClient) Write(sp storage.Peer, actions []storage.UploadAction, hostInfo *storage.HostInfo) (err error)

Source Files

View all Source files

Directories

Path Synopsis
dxdir
dxfile
Package memorymanager implements the storage client memory management methods
 Package memorymanager implements the storage client memory management methods

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.