fs533lib

package
v0.0.0-...-b744d6b Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 24, 2021 License: MIT Imports: 24 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ConvertCurrentTimeToString

func ConvertCurrentTimeToString() string

ConvertCurrentTimeToString ... to convert time.Now to a specific datetime format

func ConvertTimeToLongString

func ConvertTimeToLongString(time time.Time) string

ConvertTimeToLongString ... to convert time to a specific datetime format

Types

type Configuration

type Configuration struct {
	InstanceID                              string //instanceId of EC2 instance (a node)
	IPAddress                               string //instance's private address
	TCPPort                                 int    //TCP port which is used for listening to leave/join requests
	UDPPort                                 int    //UDP port which is used for listening to heartbeat messages
	LogPath                                 string //log files directory
	Fs533FilePath                           string //fs533 files directory
	LocalFilePath                           string //local files directory
	HostAddress                             string //host address which is IPAddress:TCPPort
	HeartbeatAddress                        string //heartbeat address which is IPAddress:UDPPort
	MaxDatagramSize                         int
	Timeout                                 int      //timeout for Join/Leave request
	HeartbeatIntervalTimeInMilliseconds     int      //interval time between 2 heartbeat messages
	HeartbeatTimeoutInMilliseconds          int      //timeout indicating that the node has failed
	GatewayNodes                            []string //3 leading nodes which are supposed that they will not fail at the same
	NumberOfPrecessorsAndSuccessors         int      //number of the predecessors and successors on the healthcheck topology ring
	NumberOfReplicas                        int      //number of preference nodes where storing replicate of a file
	WriteQuorum                             int      //write quorum
	ReadQuorum                              int      //read quorum
	FileChunkSize                           int      //chunk size for treaming file
	PendingWriteTransactionTimeoutInSeconds int      //a pending write transaction will be timeout if it reaches this total time
	NumberOfHeartbeatsShown                 int      //number of heartbeats which are printed out
	HearbeatLog                             bool     //turn on/off heartbeat log
	Membershiplog                           bool     //turn on/off membership log
}

Configuration ...Server settings

func LoadConfiguration

func LoadConfiguration(configFile string) Configuration

LoadConfiguration ... do

type FetchedFileMetadata

type FetchedFileMetadata struct {
	Fs533FileName    string //file name on the fs533 file system
	Fs533FileVersion int32  //file version
	Fs533FileSize    int32  //file size
	LocalFileName    string //file name which is fetched on local storag
}

FetchedFileMetadata ... this struct is to describe all information related to a fs533file which is already fetched to local storage

type FileClient

type FileClient struct{}

FileClient ... this struct is to handle all the file operations from client side

func (*FileClient) ClearLocalDbTestFiles

func (a *FileClient) ClearLocalDbTestFiles()

ClearLocalDbTestFiles ... This function is to delete all local test files

func (*FileClient) DeleteAllFiles

func (a *FileClient) DeleteAllFiles()

DeleteAllFiles ... This function is to delete all files

func (*FileClient) Fetch

func (a *FileClient) Fetch(fs533filename string, localfilename string)

Fetch ... this function is to list all files in fs533

func (*FileClient) Initialize

func (a *FileClient) Initialize(mn string, config Configuration)

Initialize ... setup fileClient with a given configuation file

func (*FileClient) InternalEndWriteFile

func (a *FileClient) InternalEndWriteFile(requestID string, fs533filename string, fs533filesize int32, targetNodeAddress string, cancelWriteRegistration bool, commit bool) bool

InternalEndWriteFile ... this function is to finish a write transaction, it will be commit or aborted arguments -fs533filename: filename on fs533 system which is being processed -targetNode: the node receives this commit request -cancelWriteRegistration: cancel pending write registration on master node -commit: the transation will be commited if it is true. Otherwise, the transaction will be aborted

func (*FileClient) InternalReadAcquireFile

func (a *FileClient) InternalReadAcquireFile(fs533filename string, replicaNodeAddress string) (bool, int32, []string)

InternalReadAcquireFile ... this function is to send read acquire request to the target node to get the file's latest version It return file's latest version on the target replica node and its preference nodes

func (*FileClient) InternalRegisterWriteFile

func (a *FileClient) InternalRegisterWriteFile(requestID string, fs533filename string, targetNodeAddress string) (bool, bool, int32, []string)

InternalRegisterWriteFile ... this function is to register write operation to master it should return register status, needUserInteraction, fileVersion, preferenceNodes

func (*FileClient) InternalRemoveFile

func (a *FileClient) InternalRemoveFile(fs533filename string, targetNode string) bool

InternalRemoveFile ... this function is to send remove request directly to the target node

func (*FileClient) ListAllFiles

func (a *FileClient) ListAllFiles() []string

ListAllFiles ... this function is to list all files in fs533

func (*FileClient) ListHere

func (a *FileClient) ListHere()

ListHere ... this function is to list all files in fs533

func (*FileClient) LocateFile

func (a *FileClient) LocateFile(fs533filename string) Fs533FileMetadata

LocateFile ... this function is to locate replica node for fs533 file

func (*FileClient) PrintAllFiles

func (a *FileClient) PrintAllFiles(files []string)

PrintAllFiles ... this function is to print all files queried from ListAllFile API

func (*FileClient) PrintLocateFile

func (a *FileClient) PrintLocateFile(fs533filename string)

PrintLocateFile ... this function is to list all machines (name / id / IP address) of the servers that contain a copy of the file.

func (*FileClient) Put

func (a *FileClient) Put(localfilename string, fs533filename string)

Put ... this function is to put all files in fs533

func (*FileClient) RemoveFile

func (a *FileClient) RemoveFile(fs533filename string)

RemoveFile ... this public function is to delete file fs533filename from fs533

func (*FileClient) Report

func (a *FileClient) Report()

Report ... this function is to list all the local fetched files, local replicated files and global files metadata

type FileOperation

type FileOperation struct{}

FileOperation ... this struct is to handle all the local file operations. All local files are stored on ../fs533db

func (*FileOperation) ClearFs533Db

func (a *FileOperation) ClearFs533Db()

ClearFs533Db ... delete all existing files on the fs533 db folder

func (*FileOperation) ClearLocalDb

func (a *FileOperation) ClearLocalDb(prefix string)

ClearLocalDb ... delete all existing files on the fs533 db folder

func (*FileOperation) Copy

func (*FileOperation) Copy(src string, des string) (int64, error)

Copy ... copy a file to another file

func (*FileOperation) CreateOrOpenFile

func (*FileOperation) CreateOrOpenFile(localfile string) *os.File

CreateOrOpenFile ... open or create a file for writing

func (*FileOperation) Delete

func (*FileOperation) Delete(fs533FileName string)

Delete ... delete file from folder fs533db

func (*FileOperation) Fs533FilePath

func (*FileOperation) Fs533FilePath(filename string) string

Fs533FilePath ... the path and the file name to the full path file

func (*FileOperation) ListHere

func (*FileOperation) ListHere(path string) []string

ListHere ... list all files on folder fs533db

func (*FileOperation) LocalFileExist

func (a *FileOperation) LocalFileExist(fileName string) bool

LocalFileExist ... check if the file exists on local folder or not

func (*FileOperation) LocalFilePath

func (*FileOperation) LocalFilePath(filename string) string

LocalFilePath ... the path and the file name to the full path file

func (*FileOperation) OpenFile

func (*FileOperation) OpenFile(localfile string) *os.File

OpenFile ... open file for reading

type FileServer

type FileServer struct{}

FileServer ... this struct is to handle all the file operations from server side

func (*FileServer) Backup

func (a *FileServer) Backup()

Backup ... backup all infomration about file operations that a master node should know

func (*FileServer) FireNodeLeave

func (a *FileServer) FireNodeLeave(nodeleaveAddress string)

FireNodeLeave ... farewell a node leave by handover its replicated files to the other nodes

func (*FileServer) Start

func (a *FileServer) Start(master string, eTerm int32, initFs533Files []Fs533FileMetadata, initPendingWriteTransactions []WriteTransaction)

Start ... start file server by initialize the global variables

func (*FileServer) StartLeaderElectionProcess

func (a *FileServer) StartLeaderElectionProcess()

StartLeaderElectionProcess ... start self voting process to be the new master node

func (*FileServer) StartPromotingAsMaster

func (a *FileServer) StartPromotingAsMaster() bool

StartPromotingAsMaster ... start sending requests to other nodes to self-promote as a new master node

func (*FileServer) Stop

func (a *FileServer) Stop()

Stop ... stop the current service by resetting all global variable

func (*FileServer) UpdateFs533ReplicaNodes

func (a *FileServer) UpdateFs533ReplicaNodes(newMember string)

UpdateFs533ReplicaNodes ... update all the information of file operations that a master node should know

func (*FileServer) UpdateMetada

func (a *FileServer) UpdateMetada(initFs533Files []Fs533FileMetadata, initPendingWriteTransactions []WriteTransaction)

UpdateMetada ... update all the information of file operations that a master node should know

type Fs533FileMetadata

type Fs533FileMetadata struct {
	Fs533FileName    string    //file name on the fs533 file system
	Fs533FileVersion int32     //file version
	Fs533FileSize    int32     //file size
	RegisteredTime   time.Time //latest registered time
	PreferenceNodes  []string  //list of its replica nodes
}

Fs533FileMetadata ... this truct is to describe all the information of a file located on fs533 system

type LogManager

type LogManager struct{}

LogManager ... this struct is to manage all log events

func (*LogManager) DisableLog

func (a *LogManager) DisableLog()

DisableLog ... this function is to disable logs

func (*LogManager) EnableLog

func (a *LogManager) EnableLog()

EnableLog ... this function is to enable logs related to file operation, it will log on both terminal and file

type MembershipService

type MembershipService struct {
	// contains filtered or unexported fields
}

MembershipService ... is a struct to expose public functions

func (*MembershipService) GetFollowerAddresses

func (a *MembershipService) GetFollowerAddresses() []string

GetFollowerAddresses ... This function is to return active members on the group, except the master node

func (*MembershipService) GetNodeAddresses

func (a *MembershipService) GetNodeAddresses() []string

GetNodeAddresses ... This function is to return active members on the group, except the master node

func (*MembershipService) Initialize

func (a *MembershipService) Initialize(byAddress []string, byName []string)

Initialize ... This function is to initialize the active members list arguments -byAddress: list of node ip addresses of all members -byName: list of node name including ip address and a timestamp of all members

func (*MembershipService) Len

func (a *MembershipService) Len() int

Len ... This function is to return total active members on the group

func (*MembershipService) Report

func (a *MembershipService) Report()

Report ... report current active nodes on the membership

func (*MembershipService) SetConfiguration

func (a *MembershipService) SetConfiguration(config Configuration)

SetConfiguration ... This function is to initialize the configuration

func (*MembershipService) Start

func (a *MembershipService) Start(init bool)

Start ... start membership service Arguments - init: boolean. It will start a new membership group if init is true. Otherwise, it will send joining request to the gateway nodes

func (*MembershipService) Stop

func (a *MembershipService) Stop()

Stop ... stop membership service

type MonitorPacket

type MonitorPacket struct {
	NodeAddress      string
	LastTimeReceived time.Time
	Count            int32
}

MonitorPacket ...Heartbeat package

type TemporaryFs533File

type TemporaryFs533File struct {
	Fs533FileName     string //the file being written
	Fs533FileVersion  int32  //the new version of this file
	RequestID         string //id of the write request which is to differentiate write transaction
	TemporaryFileName string //the file will be saved as a temporary file while it is waiting for the commit request
	Fs533FileSize     int32  //file size
}

TemporaryFs533File ... this struct is to describe the temporary file which is created for write preparation

type WriteTransaction

type WriteTransaction struct {
	Fs533FileName    string   //the file being written
	Fs533FileVersion int32    //the new version of this file
	RequestID        string   //id of the write request which is to differentiate write transaction
	PreferenceNodes  []string //list of its replica nodes
	RegisteredTime   time.Time
}

WriteTransaction ... this struct is to describe a pendding write transaction

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL