torrent: github.com/anacrolix/torrent Index | Examples | Files | Directories

package torrent

import "github.com/anacrolix/torrent"

Package torrent implements a torrent client. Goals include:

* Configurable data storage, such as file, mmap, and piece-based.
* Downloading on demand: torrent.Reader will request only the data required to
  satisfy Reads, which is ideal for streaming and torrentfs.

BitTorrent features implemented include:

* Protocol obfuscation
* DHT
* uTP
* PEX
* Magnet links
* IP Blocklists
* Some IPv6
* HTTP and UDP tracker clients
* BEPs:
 -  3: Basic BitTorrent protocol
 -  5: DHT
 -  6: Fast Extension (have all/none only)
 -  7: IPv6 Tracker Extension
 -  9: ut_metadata
 - 10: Extension protocol
 - 11: PEX
 - 12: Multitracker metadata extension
 - 15: UDP Tracker Protocol
 - 20: Peer ID convention ("-GTnnnn-")
 - 23: Tracker Returns Compact Peer Lists
 - 27: Private torrents
 - 29: uTorrent transport protocol
 - 41: UDP Tracker Protocol Extensions
 - 42: DHT Security extension
 - 43: Read-only DHT Nodes

Code:

c, _ := torrent.NewClient(nil)
defer c.Close()
t, _ := c.AddMagnet("magnet:?xt=urn:btih:ZOCMZQIPFFW7OLLMIC5HUB6BPCSDEOQU")
<-t.GotInfo()
t.DownloadAll()
c.WaitAll()
log.Print("ermahgerd, torrent downloaded")

Code:

var f torrent.File
// Accesses the parts of the torrent pertaining to f. Data will be
// downloaded as required, per the configuration of the torrent.Reader.
r := f.NewReader()
defer r.Close()

Index

Examples

Package Files

Peer.go Peers.go bad_storage.go bep40.go client.go closewrapper.go config.go conn_stats.go connection.go doc.go file.go global.go handshake.go listen.go misc.go multiless.go networks.go peerid.go piece.go piecestate.go portfwd.go prioritized_peers.go protocol.go ratelimitreader.go reader.go socket.go spec.go t.go torrent.go torrent_stats.go tracker_scraper.go utp.go utp_libutp.go worst_conns.go

Constants

const (
    PiecePriorityNone      piecePriority = iota // Not wanted. Must be the zero value.
    PiecePriorityNormal                         // Wanted.
    PiecePriorityHigh                           // Wanted a lot.
    PiecePriorityReadahead                      // May be required soon.
    // Succeeds a piece where a read occurred. Currently the same as Now,
    // apparently due to issues with caching.
    PiecePriorityNext
    PiecePriorityNow // A Reader is reading in this piece. Highest urgency.
)

Variables

var DefaultHTTPUserAgent = "Go-Torrent/1.0"

func LoopbackListenHost Uses

func LoopbackListenHost(network string) string

func NewUtpSocket Uses

func NewUtpSocket(network, addr string, fc firewallCallback) (utpSocket, error)

type Client Uses

type Client struct {
    // contains filtered or unexported fields
}

Clients contain zero or more Torrents. A Client manages a blocklist, the TCP/UDP protocol ports, and DHT as desired.

func NewClient Uses

func NewClient(cfg *ClientConfig) (cl *Client, err error)

func (*Client) AddDHTNodes Uses

func (cl *Client) AddDHTNodes(nodes []string)

func (*Client) AddMagnet Uses

func (cl *Client) AddMagnet(uri string) (T *Torrent, err error)

func (*Client) AddTorrent Uses

func (cl *Client) AddTorrent(mi *metainfo.MetaInfo) (T *Torrent, err error)

func (*Client) AddTorrentFromFile Uses

func (cl *Client) AddTorrentFromFile(filename string) (T *Torrent, err error)

func (*Client) AddTorrentInfoHash Uses

func (cl *Client) AddTorrentInfoHash(infoHash metainfo.Hash) (t *Torrent, new bool)

func (*Client) AddTorrentInfoHashWithStorage Uses

func (cl *Client) AddTorrentInfoHashWithStorage(infoHash metainfo.Hash, specStorage storage.ClientImpl) (t *Torrent, new bool)

Adds a torrent by InfoHash with a custom Storage implementation. If the torrent already exists then this Storage is ignored and the existing torrent returned with `new` set to `false`

func (*Client) AddTorrentSpec Uses

func (cl *Client) AddTorrentSpec(spec *TorrentSpec) (t *Torrent, new bool, err error)

Add or merge a torrent spec. If the torrent is already present, the trackers will be merged with the existing ones. If the Info isn't yet known, it will be set. The display name is replaced if the new spec provides one. Returns new if the torrent wasn't already in the client. Note that any `Storage` defined on the spec will be ignored if the torrent is already present (i.e. `new` return value is `true`)

func (*Client) BadPeerIPs Uses

func (cl *Client) BadPeerIPs() []string

func (*Client) Close Uses

func (cl *Client) Close()

Stops the client. All connections to peers are closed and all activity will come to a halt.

func (*Client) Closed Uses

func (cl *Client) Closed() <-chan struct{}

func (*Client) DhtServers Uses

func (cl *Client) DhtServers() []*dht.Server

func (*Client) ListenAddrs Uses

func (cl *Client) ListenAddrs() (ret []net.Addr)

func (*Client) LocalPort Uses

func (cl *Client) LocalPort() (port int)

func (*Client) PeerID Uses

func (cl *Client) PeerID() PeerID

func (*Client) Torrent Uses

func (cl *Client) Torrent(ih metainfo.Hash) (t *Torrent, ok bool)

Returns a handle to the given torrent, if it's present in the client.

func (*Client) Torrents Uses

func (cl *Client) Torrents() []*Torrent

Returns handles to all the torrents loaded in the Client.

func (*Client) WaitAll Uses

func (cl *Client) WaitAll() bool

Returns true when all torrents are completely downloaded and false if the client is stopped before that.

func (*Client) WriteStatus Uses

func (cl *Client) WriteStatus(_w io.Writer)

Writes out a human readable status of the client, such as for writing to a HTTP status page.

type ClientConfig Uses

type ClientConfig struct {
    // Store torrent file data in this directory unless .DefaultStorage is
    // specified.
    DataDir string `long:"data-dir" description:"directory to store downloaded torrent data"`
    // The address to listen for new uTP and TCP bittorrent protocol
    // connections. DHT shares a UDP socket with uTP unless configured
    // otherwise.
    ListenHost              func(network string) string
    ListenPort              int
    NoDefaultPortForwarding bool
    // Don't announce to trackers. This only leaves DHT to discover peers.
    DisableTrackers bool `long:"disable-trackers"`
    DisablePEX      bool `long:"disable-pex"`

    // Don't create a DHT.
    NoDHT            bool `long:"disable-dht"`
    DhtStartingNodes dht.StartingNodesGetter
    // Never send chunks to peers.
    NoUpload bool `long:"no-upload"`
    // Disable uploading even when it isn't fair.
    DisableAggressiveUpload bool `long:"disable-aggressive-upload"`
    // Upload even after there's nothing in it for us. By default uploading is
    // not altruistic, we'll only upload to encourage the peer to reciprocate.
    Seed bool `long:"seed"`
    // Only applies to chunks uploaded to peers, to maintain responsiveness
    // communicating local Client state to peers. Each limiter token
    // represents one byte. The Limiter's burst must be large enough to fit a
    // whole chunk, which is usually 16 KiB (see TorrentSpec.ChunkSize).
    UploadRateLimiter *rate.Limiter
    // Rate limits all reads from connections to peers. Each limiter token
    // represents one byte. The Limiter's burst must be bigger than the
    // largest Read performed on a the underlying rate-limiting io.Reader
    // minus one. This is likely to be the larger of the main read loop buffer
    // (~4096), and the requested chunk size (~16KiB, see
    // TorrentSpec.ChunkSize).
    DownloadRateLimiter *rate.Limiter

    // User-provided Client peer ID. If not present, one is generated automatically.
    PeerID string
    // For the bittorrent protocol.
    DisableUTP bool
    // For the bittorrent protocol.
    DisableTCP bool `long:"disable-tcp"`
    // Called to instantiate storage for each added torrent. Builtin backends
    // are in the storage package. If not set, the "file" implementation is
    // used.
    DefaultStorage storage.ClientImpl

    HeaderObfuscationPolicy HeaderObfuscationPolicy
    // The crypto methods to offer when initiating connections with header obfuscation.
    CryptoProvides mse.CryptoMethod
    // Chooses the crypto method to use when receiving connections with header obfuscation.
    CryptoSelector mse.CryptoSelector

    // Sets usage of Socks5 Proxy. Authentication should be included in the url if needed.
    // Examples: socks5://demo:demo@192.168.99.100:1080
    // 			 http://proxy.domain.com:3128
    ProxyURL string

    IPBlocklist      iplist.Ranger
    DisableIPv6      bool `long:"disable-ipv6"`
    DisableIPv4      bool
    DisableIPv4Peers bool
    // Perform logging and any other behaviour that will help debug.
    Debug  bool `help:"enable debugging"`
    Logger log.Logger

    // HTTPProxy defines proxy for HTTP requests.
    // Format: func(*Request) (*url.URL, error),
    // or result of http.ProxyURL(HTTPProxy).
    // By default, it is composed from ClientConfig.ProxyURL,
    // if not set explicitly in ClientConfig struct
    HTTPProxy func(*http.Request) (*url.URL, error)
    // HTTPUserAgent changes default UserAgent for HTTP requests
    HTTPUserAgent string
    // Updated occasionally to when there's been some changes to client
    // behaviour in case other clients are assuming anything of us. See also
    // `bep20`.
    ExtendedHandshakeClientVersion string
    // Peer ID client identifier prefix. We'll update this occasionally to
    // reflect changes to client behaviour that other clients may depend on.
    // Also see `extendedHandshakeClientVersion`.
    Bep20 string

    // Peer dial timeout to use when there are limited peers.
    NominalDialTimeout time.Duration
    // Minimum peer dial timeout to use (even if we have lots of peers).
    MinDialTimeout             time.Duration
    EstablishedConnsPerTorrent int
    HalfOpenConnsPerTorrent    int
    // Maximum number of peer addresses in reserve.
    TorrentPeersHighWater int
    // Minumum number of peers before effort is made to obtain more peers.
    TorrentPeersLowWater int

    // Limit how long handshake can take. This is to reduce the lingering
    // impact of a few bad apples. 4s loses 1% of successful handshakes that
    // are obtained with 60s timeout, and 5% of unsuccessful handshakes.
    HandshakesTimeout time.Duration

    // The IP addresses as our peers should see them. May differ from the
    // local interfaces due to NAT or other network configurations.
    PublicIp4 net.IP
    PublicIp6 net.IP

    DisableAcceptRateLimiting bool

    ConnTracker *conntrack.Instance

    // OnQuery hook func
    DHTOnQuery func(query *krpc.Msg, source net.Addr) (propagate bool)
    // contains filtered or unexported fields
}

Probably not safe to modify this after it's given to a Client.

func NewDefaultClientConfig Uses

func NewDefaultClientConfig() *ClientConfig

func (*ClientConfig) SetListenAddr Uses

func (cfg *ClientConfig) SetListenAddr(addr string) *ClientConfig

type ConnStats Uses

type ConnStats struct {
    // Total bytes on the wire. Includes handshakes and encryption.
    BytesWritten     Count
    BytesWrittenData Count

    BytesRead           Count
    BytesReadData       Count
    BytesReadUsefulData Count

    ChunksWritten Count

    ChunksRead       Count
    ChunksReadUseful Count
    ChunksReadWasted Count

    MetadataChunksRead Count

    // Number of pieces data was written to, that subsequently passed verification.
    PiecesDirtiedGood Count
    // Number of pieces data was written to, that subsequently failed
    // verification. Note that a connection may not have been the sole dirtier
    // of a piece.
    PiecesDirtiedBad Count
}

Various connection-level metrics. At the Torrent level these are aggregates. Chunks are messages with data payloads. Data is actual torrent content without any overhead. Useful is something we needed locally. Unwanted is something we didn't ask for (but may still be useful). Written is things sent to the peer, and Read is stuff received from them.

func (*ConnStats) Copy Uses

func (me *ConnStats) Copy() (ret ConnStats)

type Count Uses

type Count struct {
    // contains filtered or unexported fields
}

func (*Count) Add Uses

func (me *Count) Add(n int64)

func (*Count) Int64 Uses

func (me *Count) Int64() int64

func (*Count) MarshalJSON Uses

func (me *Count) MarshalJSON() ([]byte, error)

func (*Count) String Uses

func (me *Count) String() string

type File Uses

type File struct {
    // contains filtered or unexported fields
}

Provides access to regions of torrent data that correspond to its files.

func (*File) Cancel Uses

func (f *File) Cancel()

Deprecated: Use File.SetPriority.

func (*File) DisplayPath Uses

func (f *File) DisplayPath() string

The relative file path for a multi-file torrent, and the torrent name for a single-file torrent.

func (*File) Download Uses

func (f *File) Download()

Requests that all pieces containing data in the file be downloaded.

func (File) FileInfo Uses

func (f File) FileInfo() metainfo.FileInfo

The FileInfo from the metainfo.Info to which this file corresponds.

func (*File) Length Uses

func (f *File) Length() int64

The file's length in bytes.

func (*File) NewReader Uses

func (f *File) NewReader() Reader

func (*File) Offset Uses

func (f *File) Offset() int64

Data for this file begins this many bytes into the Torrent.

func (File) Path Uses

func (f File) Path() string

The file's path components joined by '/'.

func (*File) Priority Uses

func (f *File) Priority() piecePriority

Returns the priority per File.SetPriority.

func (*File) SetPriority Uses

func (f *File) SetPriority(prio piecePriority)

Sets the minimum priority for pieces in the File.

func (*File) State Uses

func (f *File) State() (ret []FilePieceState)

Returns the state of pieces in this file.

func (*File) Torrent Uses

func (f *File) Torrent() *Torrent

type FilePieceState Uses

type FilePieceState struct {
    Bytes int64 // Bytes within the piece that are part of this File.
    PieceState
}

The download status of a piece that comprises part of a File.

type Handle Uses

type Handle interface {
    io.Reader
    io.Seeker
    io.Closer
    io.ReaderAt
}

A file-like handle to some torrent data resource.

type HeaderObfuscationPolicy Uses

type HeaderObfuscationPolicy struct {
    RequirePreferred bool // Whether the value of Preferred is a strict requirement.
    Preferred        bool // Whether header obfuscation is preferred.
}

type InfoHash Uses

type InfoHash = metainfo.Hash

type IpPort Uses

type IpPort = missinggo.IpPort

type Peer Uses

type Peer struct {
    Id     [20]byte
    IP     net.IP
    Port   int
    Source peerSource
    // Peer is known to support encryption.
    SupportsEncryption bool
    peer_protocol.PexPeerFlags
}

func (*Peer) FromPex Uses

func (me *Peer) FromPex(na krpc.NodeAddr, fs peer_protocol.PexPeerFlags)

type PeerExtensionBits Uses

type PeerExtensionBits = pp.PeerExtensionBits

type PeerID Uses

type PeerID [20]byte

Peer client ID.

type Peers Uses

type Peers []Peer

func (*Peers) AppendFromPex Uses

func (me *Peers) AppendFromPex(nas []krpc.NodeAddr, fs []peer_protocol.PexPeerFlags)

func (Peers) AppendFromTracker Uses

func (ret Peers) AppendFromTracker(ps []tracker.Peer) Peers

type Piece Uses

type Piece struct {
    // contains filtered or unexported fields
}

func (*Piece) Info Uses

func (p *Piece) Info() metainfo.Piece

func (*Piece) SetPriority Uses

func (p *Piece) SetPriority(prio piecePriority)

func (*Piece) Storage Uses

func (p *Piece) Storage() storage.Piece

func (*Piece) String Uses

func (p *Piece) String() string

func (*Piece) VerifyData Uses

func (p *Piece) VerifyData()

Forces the piece data to be rehashed.

type PieceState Uses

type PieceState struct {
    Priority piecePriority
    storage.Completion
    // The piece is being hashed, or is queued for hash.
    Checking bool
    // Some of the piece has been obtained.
    Partial bool
}

The current state of a piece.

type PieceStateChange Uses

type PieceStateChange struct {
    Index int
    PieceState
}

type PieceStateRun Uses

type PieceStateRun struct {
    PieceState
    Length int // How many consecutive pieces have this state.
}

Represents a series of consecutive pieces with the same state.

type Reader Uses

type Reader interface {
    io.Reader
    io.Seeker
    io.Closer
    missinggo.ReadContexter
    SetReadahead(int64)
    SetResponsive()
}

type Torrent Uses

type Torrent struct {
    // contains filtered or unexported fields
}

Maintains state of torrent within a Client.

func (*Torrent) AddClientPeer Uses

func (t *Torrent) AddClientPeer(cl *Client)

func (*Torrent) AddPeers Uses

func (t *Torrent) AddPeers(pp []Peer)

func (*Torrent) AddTrackers Uses

func (t *Torrent) AddTrackers(announceList [][]string)

func (*Torrent) BytesCompleted Uses

func (t *Torrent) BytesCompleted() int64

Number of bytes of the entire torrent we have completed. This is the sum of completed pieces, and dirtied chunks of incomplete pieces. Do not use this for download rate, as it can go down when pieces are lost or fail checks. Sample Torrent.Stats.DataBytesRead for actual file data download rate.

func (*Torrent) BytesMissing Uses

func (t *Torrent) BytesMissing() int64

func (*Torrent) CancelPieces Uses

func (t *Torrent) CancelPieces(begin, end pieceIndex)

func (*Torrent) Closed Uses

func (t *Torrent) Closed() <-chan struct{}

Returns a channel that is closed when the Torrent is closed.

func (*Torrent) DownloadAll Uses

func (t *Torrent) DownloadAll()

Marks the entire torrent for download. Requires the info first, see GotInfo. Sets piece priorities for historical reasons.

func (*Torrent) DownloadPieces Uses

func (t *Torrent) DownloadPieces(begin, end pieceIndex)

Raise the priorities of pieces in the range [begin, end) to at least Normal priority. Piece indexes are not the same as bytes. Requires that the info has been obtained, see Torrent.Info and Torrent.GotInfo.

func (*Torrent) Drop Uses

func (t *Torrent) Drop()

Drop the torrent from the client, and close it. It's always safe to do this. No data corruption can, or should occur to either the torrent's data, or connected peers.

func (*Torrent) Files Uses

func (t *Torrent) Files() []*File

Returns handles to the files in the torrent. This requires that the Info is available first.

func (*Torrent) GotInfo Uses

func (t *Torrent) GotInfo() <-chan struct{}

Returns a channel that is closed when the info (.Info()) for the torrent has become available.

func (*Torrent) Info Uses

func (t *Torrent) Info() *metainfo.Info

Returns the metainfo info dictionary, or nil if it's not yet available.

func (*Torrent) InfoHash Uses

func (t *Torrent) InfoHash() metainfo.Hash

The torrent's infohash. This is fixed and cannot change. It uniquely identifies a torrent.

func (*Torrent) KnownSwarm Uses

func (t *Torrent) KnownSwarm() (ks []Peer)

KnownSwarm returns the known subset of the peers in the Torrent's swarm, including active, pending, and half-open peers.

func (*Torrent) Length Uses

func (t *Torrent) Length() int64

The completed length of all the torrent data, in all its files. This is derived from the torrent info, when it is available.

func (*Torrent) Metainfo Uses

func (t *Torrent) Metainfo() metainfo.MetaInfo

Returns a run-time generated metainfo for the torrent that includes the info bytes and announce-list as currently known to the client.

func (*Torrent) Name Uses

func (t *Torrent) Name() string

The current working name for the torrent. Either the name in the info dict, or a display name given such as by the dn value in a magnet link, or "".

func (*Torrent) NewReader Uses

func (t *Torrent) NewReader() Reader

Returns a Reader bound to the torrent's data. All read calls block until the data requested is actually available.

func (*Torrent) NumPieces Uses

func (t *Torrent) NumPieces() pieceIndex

The number of pieces in the torrent. This requires that the info has been obtained first.

func (*Torrent) Piece Uses

func (t *Torrent) Piece(i pieceIndex) *Piece

func (*Torrent) PieceBytesMissing Uses

func (t *Torrent) PieceBytesMissing(piece int) int64

Get missing bytes count for specific piece.

func (*Torrent) PieceState Uses

func (t *Torrent) PieceState(piece pieceIndex) PieceState

func (*Torrent) PieceStateRuns Uses

func (t *Torrent) PieceStateRuns() []PieceStateRun

Returns the state of pieces of the torrent. They are grouped into runs of same state. The sum of the state run lengths is the number of pieces in the torrent.

func (*Torrent) Seeding Uses

func (t *Torrent) Seeding() bool

Returns true if the torrent is currently being seeded. This occurs when the client is willing to upload without wanting anything in return.

func (*Torrent) SetDisplayName Uses

func (t *Torrent) SetDisplayName(dn string)

Clobbers the torrent display name. The display name is used as the torrent name if the metainfo is not available.

func (*Torrent) SetInfoBytes Uses

func (t *Torrent) SetInfoBytes(b []byte) (err error)

func (*Torrent) SetMaxEstablishedConns Uses

func (t *Torrent) SetMaxEstablishedConns(max int) (oldMax int)

func (*Torrent) Stats Uses

func (t *Torrent) Stats() TorrentStats

func (*Torrent) String Uses

func (t *Torrent) String() string

func (*Torrent) SubscribePieceStateChanges Uses

func (t *Torrent) SubscribePieceStateChanges() *pubsub.Subscription

The subscription emits as (int) the index of pieces as their state changes. A state change is when the PieceState for a piece alters in value.

func (*Torrent) VerifyData Uses

func (t *Torrent) VerifyData()

Forces all the pieces to be re-hashed. See also Piece.VerifyData.

type TorrentSpec Uses

type TorrentSpec struct {
    // The tiered tracker URIs.
    Trackers  [][]string
    InfoHash  metainfo.Hash
    InfoBytes []byte
    // The name to use if the Name field from the Info isn't available.
    DisplayName string
    // The chunk size to use for outbound requests. Defaults to 16KiB if not
    // set.
    ChunkSize int
    Storage   storage.ClientImpl
}

Specifies a new torrent for adding to a client. There are helpers for magnet URIs and torrent metainfo files.

func TorrentSpecFromMagnetURI Uses

func TorrentSpecFromMagnetURI(uri string) (spec *TorrentSpec, err error)

func TorrentSpecFromMetaInfo Uses

func TorrentSpecFromMetaInfo(mi *metainfo.MetaInfo) (spec *TorrentSpec)

type TorrentStats Uses

type TorrentStats struct {
    // Aggregates stats over all connections past and present. Some values may
    // not have much meaning in the aggregate context.
    ConnStats

    // Ordered by expected descending quantities (if all is well).
    TotalPeers       int
    PendingPeers     int
    ActivePeers      int
    ConnectedSeeders int
    HalfOpenPeers    int
}

Directories

PathSynopsis
bencode
cmd/magnet-metainfoConverts magnet URIs and info hashes into torrent metainfo files.
cmd/torrentDownloads torrents from the command-line.
cmd/torrent-create
cmd/torrentfsMounts a FUSE filesystem backed by torrents and magnet links.
cmd/torrent-infohash
cmd/torrent-magnet
cmd/torrent-metainfo-pprint
cmd/torrent-pickDownloads torrents from the command-line.
cmd/torrent-verify
cmd/tracker-announce
fs
internal/testutil
iplistPackage iplist handles the P2P Plaintext Format described by https://en.wikipedia.org/wiki/PeerGuardian#P2P_plaintext_format.
iplist/cmd/iplist
iplist/cmd/pack-blocklistTakes P2P blocklist text format in stdin, and outputs the packed format from the iplist package.
logoncePackage logonce implements an io.Writer facade that only performs distinct writes.
metainfo
mmap_span
mse
peer_protocol
storagePackage storage implements storage backends for package torrent.
tracker
util/dirwatchPackage dirwatch provides filesystem-notification based tracking of torrent info files and magnet URIs in a directory.

Package torrent imports 58 packages (graph) and is imported by 78 packages. Updated 2019-09-13. Refresh now. Tools for package owners.