README
¶
uspv - micro-SPV library
The uspv library implements simplified SPV wallet functionality. It connects to full nodes using the standard port 8333 bitcoin protocol, gets headers, uses bloom filters, gets blocks and transactions, and has functions to send and receive coins.
Files
Three files are used by the library:
Key file (currently testkey.hex)
This file contains the secret seed which creates all private keys used by the wallet. It is stored in ascii hexadecimal for easy copying and pasting. If you don't enter a password when prompted, you'll get a warning and the key file will be saved in the clear with no encryption. You shouldn't do that though. When using a password, the key file will be longer and use the scrypt KDF and nacl/secretbox to protect the secret seed.
Header file (currently headers.bin)
This is a file storing all the block headers. Headers are 80 bytes long, so this file's size will always be an even multiple of 80. All blockchain-technology verifications are performed when appending headers to the file. In the case of re-orgs, since it's so quick to get headers, it just truncates a bit and tries again.
Database file (currently utxo.db)
This file more complex. It uses bolt DB to store wallet information needed to send and receive bitcoins. The database file is organized into 4 main "buckets":
- Utxos ("DuffelBag")
This bucket stores all the utxos. The goal of bitcoin is to get lots of utxos, earning a high score.
- Stxos ("SpentTxs")
For record keeping, this bucket stores what used to be utxos, but are no longer "u"txos, and are spent outpoints. It references the spending txid.
- Txns ("Txns")
This bucket stores full serialized transactions which are refenced in the Stxos bucket. These can be used to re-play transactions in the case of re-orgs.
- State ("MiscState")
This has describes some miscellaneous global state variables of the database, such as what height it has synchronized up to, and how many addresses have been created. (Currently those are the only 2 things stored)
Synchronization overview
Currently uspv only connects to one hard-coded node, as address messages and storage are not yet implemented. It first asks for headers, providing the last known header (writing the genesis header if needed). It loops through asking for headers until it receives an empty header message, which signals that headers are fully synchronized.
After header synchronization is complete, it requests merkle blocks starting at the keyfile birthday. (This is currently hard-coded; add new db key?) Bloom filters are generated for the addresses and utxos known to the wallet. If too many false positives are received, a new filter is generated and sent. (This happens fairly often because the filter exponentially saturates with false positives when using BloomUpdateAll.) Once the merkle blocks have been received up to the header height, the wallet is considered synchronized and it will listen for new inv messages from the remote node. An inv message describing a block will trigger a request for headers, starting the same synchronization process of headers then merkle-blocks.
TODO
There's still quite a bit left, though most of it hopefully won't be too hard.
Problems / still to do:
- Only connects to one node, and that node is hard-coded.
- Re-orgs affect only headers, and don't evict confirmed transactions.
- Double spends are not detected; Double spent txs will stay at height 0.
- Tx creation and signing is still very rudimentary.
- There may be wire-protocol irregularities which can get it kicked off.
Hopefully I can get most of that list deleted soon.
(Now sanity checks txs, but can't check sigs... because it's SPV. Right.)
Later functionality to implement:
- "Desktop Mode" SPV, or "Unfiltered" SPV or some other name
This would be a mode where uspv doesn't use bloom filters and request merkle blocks, but instead grabs everything in the block and discards most of the data. This prevents nodes from learning about your utxo set. To further enhance this, it should connect to multiple nodes and relay txs and inv messages to blend in.
- Ironman SPV
Never request txs. Only merkleBlocks (or in above mode, blocks). No unconfirmed transactions are presented to the user, which makes a whole lot of sense as with unconfirmed SPV transactions you're relying completely on the honesty of the reporting node.
C4:1Lb99OK
Documentation
¶
Index ¶
- Constants
- Variables
- func BlockOK(blk wire.MsgBlock) bool
- func CheckHeaderChain(r io.ReadSeeker, inHeaders []*wire.BlockHeader, p *coinparam.Params) (int32, error)
- func FindHeader(r io.ReadSeeker, hdr wire.BlockHeader) (int32, error)
- func IP4(ipAddress string) bool
- func MakeMerkleParent(left, right *chainhash.Hash) *chainhash.Hash
- type ChainHook
- type HashAndHeight
- type SPVCon
- func (s *SPVCon) AskForBlocks() error
- func (s *SPVCon) AskForHeaders() error
- func (s *SPVCon) AskForTx(txid chainhash.Hash)
- func (s *SPVCon) Connect(remoteNode string) error
- func (s *SPVCon) DialNode(listOfNodes []string) error
- func (s *SPVCon) GetDataHandler(m *wire.MsgGetData)
- func (s *SPVCon) GetHeaderAtHeight(h int32) (*wire.BlockHeader, error)
- func (s *SPVCon) GetHeaderTipHeight() int32
- func (s *SPVCon) GetListOfNodes() ([]string, error)
- func (s *SPVCon) GimmeFilter() (*bloom.Filter, error)
- func (s *SPVCon) Handshake(listOfNodes []string) error
- func (s *SPVCon) HeaderHandler(m *wire.MsgHeaders)
- func (s *SPVCon) IngestBlock(m *wire.MsgBlock)
- func (s *SPVCon) IngestHeaders(m *wire.MsgHeaders) (bool, error)
- func (s *SPVCon) IngestMerkleBlock(m *wire.MsgMerkleBlock)
- func (s *SPVCon) InvHandler(m *wire.MsgInv)
- func (s *SPVCon) MatchTx(tx *wire.MsgTx) bool
- func (s *SPVCon) NewHeightChannel() chan int32
- func (s *SPVCon) NewRawBlocksChannel() chan *wire.MsgBlock
- func (s *SPVCon) OKTxid(txid *chainhash.Hash, height int32) error
- func (s *SPVCon) PongBack(nonce uint64)
- func (s *SPVCon) PushTx(tx *wire.MsgTx) error
- func (s *SPVCon) RawBlocks() chan *wire.MsgBlock
- func (s *SPVCon) Refilter(f *bloom.Filter)
- func (s *SPVCon) RegisterAddress(adr160 [20]byte) error
- func (s *SPVCon) RegisterOutPoint(op wire.OutPoint) error
- func (s *SPVCon) SendFilter(f *bloom.Filter)
- func (s *SPVCon) Start(startHeight int32, host, path string, proxyURL string, ...) (chan lnutil.TxAndHeight, chan int32, error)
- func (s *SPVCon) TxHandler(tx *wire.MsgTx)
- func (s *SPVCon) UnregisterOutPoint(op wire.OutPoint) error
Constants ¶
const ( // VERSION hardcoded for now, probably ok...? // 70012 is for segnet... make this an init var? VERSION = 70012 )
Variables ¶
var ( // WitMagicBytes ... WitMagicBytes = []byte{0x6a, 0x24, 0xaa, 0x21, 0xa9, 0xed} )
Functions ¶
func BlockOK ¶
BlockOK checks for block self-consistency. If the block has no wintess txs, and no coinbase witness commitment, it only checks the tx merkle root. If either a witness commitment or any witnesses are detected, it also checks that as well. Returns false if anything goes wrong, true if everything is fine.
func CheckHeaderChain ¶
func CheckHeaderChain( r io.ReadSeeker, inHeaders []*wire.BlockHeader, p *coinparam.Params) (int32, error)
CheckHeaderChain takes in the headers message and sees if they all validate. This function also needs read access to the previous headers. Does not deal with re-orgs; assumes new headers link to tip returns true if *all* headers are cool, false if there is any problem Note we don't know what the height is, just the relative height. returnin nil means it worked returns an int32 usually 0, but if there's a reorg, shows what height to reorg back to before adding on the headers
func FindHeader ¶
func FindHeader(r io.ReadSeeker, hdr wire.BlockHeader) (int32, error)
FindHeader will try to find where the header you give it is. it runs backwards to find it and gives up after 1000 headers
Types ¶
type ChainHook ¶
type ChainHook interface {
// Note that for reorgs, the height chan just sends a lower height than you
// already have, and that means "reorg back!"
Start(height int32, host, path string, proxyURL string, params *coinparam.Params) (
chan lnutil.TxAndHeight, chan int32, error)
// RegisterAddress tells the ChainHook about an address of interest.
// Give it an array; Currently needs to be 20 bytes. Only p2pkh / p2wpkh
// are supported.
// Later could add a separate function for script hashes (20/32)
RegisterAddress(address [20]byte) error
// RegisterOutPoint tells the ChainHook about an outpoint of interest.
RegisterOutPoint(wire.OutPoint) error
// UnregisterOutPoint tells the ChainHook about loss of interest in an outpoint.
UnregisterOutPoint(wire.OutPoint) error
// NewRawBlocksChannel returns a new channel to receive blocks.
NewRawBlocksChannel() chan *wire.MsgBlock
// NewHeightChannel returns a new channel to receive height events.
NewHeightChannel() chan int32
// PushTx sends a tx out to the network via the ChainHook.
// Note that this does NOT register anything in the tx, so by just using this,
// nothing will come back about confirmation. It WILL come back with errors
// though, so this takes some time.
PushTx(tx *wire.MsgTx) error
// Request all incoming blocks over this channel. If RawBlocks isn't called,
// then the undelying hook package doesn't need to get full blocks.
// Currently you always call it with uspv...
RawBlocks() chan *wire.MsgBlock
}
ChainHook is a thing that lets you interact with the actual blockchains
type HashAndHeight ¶
type HashAndHeight struct {
// contains filtered or unexported fields
}
HashAndHeight is needed instead of just height in case a fullnode responds abnormally (?) by sending out of order merkleblocks. we cache a merkleroot:height pair in the queue so we don't have to look them up from the disk. Also used when inv messages indicate blocks so we can add the header and parse the txs in one request instead of requesting headers first.
func NewRootAndHeight ¶
func NewRootAndHeight(b chainhash.Hash, h int32) (hah HashAndHeight)
NewRootAndHeight saves like 2 lines.
type SPVCon ¶
type SPVCon struct {
// Enhanced SPV modes for users who have outgrown easy mode SPV
// but have not yet graduated to full nodes.
HardMode bool // hard mode doesn't use filters.
Ironman bool // ironman only gets blocks, never requests txs.
ProxyURL string // Optionally the URL of a SOCKS5 proxy to use
OKTxids map[chainhash.Hash]int32 // known good txids and their heights
OKMutex sync.Mutex
// TrackingAdrs and OPs are slices of addresses and outpoints to watch for.
// Using struct{} saves a byte of RAM but is ugly so I'll use bool.
TrackingAdrs map[[20]byte]bool
TrackingAdrsMtx sync.Mutex
TrackingOPs map[wire.OutPoint]bool
TrackingOPsMtx sync.Mutex
// TxMap is an in-memory map of all the Txs the SPVCon knows about
TxMap map[chainhash.Hash]*wire.MsgTx
WBytes uint64 // total bytes written
RBytes uint64 // total bytes read
Param *coinparam.Params // network parameters (testnet3, segnet, etc)
// TxUpToWallit is the channel for sending txs up a level to the wallit.
TxUpToWallit chan lnutil.TxAndHeight
// CurrentHeightChan is how we tell the wallit when blocks come in
CurrentHeightChan chan int32
// RawBlockSender is a channel to send full blocks up to the qln / watchtower
// only kicks in when requested from upper layer
RawBlockSender chan *wire.MsgBlock
// If the above RawBlockSender chan isn't being pulled from, don't send to it
RawBlockActive bool
// RawBlockDistribute is a list of channels that the raw blocks get sent to.
// This is so we can have multiple things using and requesting blocks from the
// same chainhook.
RawBlockDistribute []chan *wire.MsgBlock
// HeightDistribute is a list, like RawBlockDistribute, of channels where we
// send height events
HeightDistribute []chan int32
// contains filtered or unexported fields
}
SPVCon is a SPV connection to a coin daemon.
func (*SPVCon) AskForBlocks ¶
AskForBlocks requests blocks from current to last right now this asks for 1 block per getData message. Maybe it's faster to ask for many in each message?
func (*SPVCon) AskForTx ¶
AskForTx requests a tx we heard about from an inv message. It's one at a time but should be fast enough. I don't like this function because SPV shouldn't even ask...
func (*SPVCon) Connect ¶
Connect dials out and connects to full nodes. Calls GetListOfNodes to get the list of nodes if the user has specified a YupString. Else, moves on to dial the node to see if its up and establishes a connection followed by Handshake() which sends out wire messages, checks for version string to prevent spam, etc.
func (*SPVCon) DialNode ¶
DialNode receives a list of node ips and then tries to connect to them one by one.
func (*SPVCon) GetDataHandler ¶
func (s *SPVCon) GetDataHandler(m *wire.MsgGetData)
GetDataHandler responds to requests for tx data, which happen after advertising our txs via an inv message
func (*SPVCon) GetHeaderAtHeight ¶
func (s *SPVCon) GetHeaderAtHeight(h int32) (*wire.BlockHeader, error)
GetHeaderAtHeight gives back a header at the specified height
func (*SPVCon) GetHeaderTipHeight ¶
GetHeaderTipHeight gives back a header at the specified height.
func (*SPVCon) GetListOfNodes ¶
GetListOfNodes contacts all DNSSeeds for the coin specified and then contacts each one of them in order to receive a list of ips and then returns a combined list
func (*SPVCon) GimmeFilter ¶
GimmeFilter ... or I'm gonna fade away
func (*SPVCon) IngestBlock ¶
IngestBlock is like IngestMerkleBlock but aralphic different enough that it's better to have 2 separate functions
func (*SPVCon) IngestHeaders ¶
func (s *SPVCon) IngestHeaders(m *wire.MsgHeaders) (bool, error)
IngestHeaders takes in a bunch of headers, checks them, and if they're OK, appends them to the local header file. If there are no headers, it assumes we're done and returns false. Otherwise it assumes there's more to request and returns true.
func (*SPVCon) IngestMerkleBlock ¶
func (s *SPVCon) IngestMerkleBlock(m *wire.MsgMerkleBlock)
IngestMerkleBlock ...
func (*SPVCon) NewHeightChannel ¶
NewHeightChannel appends to the height distribution list and returns a new channel to receive height events.
func (*SPVCon) NewRawBlocksChannel ¶
NewRawBlocksChannel appends to the raw block distribution list and returns a new channel to receive blocks.
func (*SPVCon) OKTxid ¶
OKTxid assigns a height to a txid. This means that the txid exists at that height, with whatever assurance (for height 0 it's no assurance at all)
func (*SPVCon) Refilter ¶
Refilter reconstructs the local in-memory bloom filter. It does this by calling GimmeFilter() but doesn't broadcast the result.
func (*SPVCon) RegisterAddress ¶
RegisterAddress ...
func (*SPVCon) RegisterOutPoint ¶
RegisterOutPoint ...
Source Files