README
¶
chihaya-privtrak-middleware
A private tracker middleware for the chihaya BitTorrent tracker.
go get -t -u github.com/mrd0ll4r/chihaya-privtrak-middleware
Features
- In-memory storage of peer stats
- Versatile user ID handling (anything unique that fits into 16 bytes works)
- Generates deltas for user/infohash pairs
- Batches deltas, hands them off to a
DeltaHandler, for further processing - Purges stale peer stats
- Built for concurrency
Non-Features
- Doesn't check whether a user is allowed to announce. Write your own middleware for that, don't try to build one monolithic private tracker middleware.
- Doesn't check whether an infohash/client/... is whitelisted. Same as above.
How to use this
the whole thing is in early stages, beware
Roughly, you need to do something like this:
- Implement a
UserIdentifierto derive unique user IDs from requests. You could use HTTP/UDP optional parameters for this, or customized routes, or ... Just make sure the process is fast and reliable. Do not throw database queries in here, you WILL kill your tracker and/or DB. A sane way of doing this would be to issue 16-byte passkeys and just use them as user IDs. - Implement a
DeltaHandlerto handle the deltas. You'd probably write these to a DB, or reprocess them first. They come in batches for a reason - don't insert them into a DB one by one. Before throwing them into a DB, you might want to- look up a user's numeric ID
- look up the torrents numeric ID
- (aggregate, possibly)
- push them into an event processing system, for cool continuous queries
- produce prometheus stats?
- Compile this middleware into your chihaya instance by adding a few lines in the
cmd/chihay/config.gofile. - Adjust your config accordingly.
Expected issues
- Should roughly double the memory usage of a tracker running optmem. It'd be elegant to merge the two somehow, but I don't see that happening soon.
- Must run as a PreHook to throttle tracker throughput in case of a slow
DeltaHandler. Otherwise we just get a goroutine explosion.
Future plans
-
Add an interface to deal with backup/restore functionality Basically, it needs to:
- Restore everything on startup
- Be notified whenever we remove a peer due to GC (maybe also stopped peers? What about partial seeds? What about paused downloads?)
- Figure out if we know a user/infohash combination before we create it from scratch
-
Possibly include IP/PeerID in the deltas? The
DeltaHandlercan figure out whether to remove these due to special user's protection.
License
MIT, see LICENSE
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
View Source
var ( // PromGCDurationMilliseconds is a histogram used by storage to record the // durations of execution time required for removing expired peers. PromGCDurationMilliseconds = prometheus.NewHistogram(prometheus.HistogramOpts{ Name: "chihaya_privtrak_gc_duration_milliseconds", Help: "The time it takes to perform privtrak garbage collection", Buckets: prometheus.ExponentialBuckets(9.375, 2, 10), }) // PromUsersCount is a gauge used to hold the current total amount of // unique swarms being tracked by a storage. PromUsersCount = prometheus.NewGauge(prometheus.GaugeOpts{ Name: "chihaya_privtrak_users_count", Help: "The number of users tracked", }) )
View Source
var ErrClosing = errors.New("privtrak middleware closing")
ErrClosing is returned on an announce if the middleware is shut down.
Functions ¶
func New ¶
func New(provided Config, identifier UserIdentifier, handler DeltaHandler) (middleware.Hook, error)
New constructs a new instance of the privtrak middleware.
Types ¶
type Config ¶
type Config struct {
BatchSize uint `yaml:"batch_size"`
ShardCount uint `yaml:"shard_count"`
GCInterval time.Duration `yaml:"gc_interval"`
PrometheusReportingInterval time.Duration `yaml:"prometheus_reporting_interval"`
PeerLifetime time.Duration `yaml:"peer_lifetime"`
}
A Config holds the configuration for the privtrak middleware.
type DeltaHandler ¶
type DeltaHandler interface {
// HandleDeltas handles a batch of deltas.
// This operation is assumed to be atomic:
// if no error is returned, the deltas are assumed to be persisted and
// thus deleted from this middleware
// if an error is returned, the deltas are kept in the middleware and the
// error will be returned on the announce route. We will attempt to
// flush the deltas again next time, or during routine garbage
// collection.
HandleDeltas([]StatDelta) error
}
A DeltaHandler handles batches of stat-deltas.
type StatDelta ¶
type StatDelta struct {
User ID
InfoHash bittorrent.InfoHash
DeltaUp int64
DeltaDown int64
Event bittorrent.Event
Reported time.Time
}
A StatDelta is a delta for a user's stats. It contains the infohash, the event the client provided with the announce and a timestamp.
type UserIdentifier ¶
type UserIdentifier interface {
DeriveID(*bittorrent.AnnounceRequest) (ID, error)
}
A UserIdentifier identifies an ID from a request.
Source Files
Click to show internal directories.
Click to hide internal directories.