mtglib

package
v2.1.7 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 9, 2022 License: MIT Imports: 20 Imported by: 1

Documentation

Overview

mtglib defines a package with MTPROTO proxy.

Since mtg itself is build as an example of how to work with mtglib, it worth to telling a couple of words about a project organization.

A core object of the project is mtglib.Proxy. This is a proxy you expect: that one which you configure, set to serve on a listener and/or shutdown on application termination.

But it also has a core logic unrelated to Telegram per se: anti replay cache, network connectivity (who knows, maybe you want to have a native VMESS integration) and so on.

You can supply such parts to a proxy with interfaces. The rest of the packages in mtg define some default implementations of these interfaces. But if you want to integrate it with, let say, influxdb, you can do it easily.

Index

Constants

View Source
const (
	// DefaultConcurrency is a default max count of simultaneously connected
	// clients.
	DefaultConcurrency = 4096

	// DefaultBufferSize is a default size of a copy buffer.
	//
	// Deprecated: this setting no longer makes any effect.
	DefaultBufferSize = 16 * 1024 // 16 kib

	// DefaultDomainFrontingPort is a default port (HTTPS) to connect to in case
	// of probe-resistance activity.
	DefaultDomainFrontingPort = 443

	// DefaultIdleTimeout is a default timeout for closing a connection in case of
	// idling.
	//
	// Deprecated: no longer in use because of changed TCP relay algorithm.
	DefaultIdleTimeout = time.Minute

	// DefaultTolerateTimeSkewness is a default timeout for time skewness on a
	// faketls timeout verification.
	DefaultTolerateTimeSkewness = 3 * time.Second

	// DefaultPreferIP is a default value for Telegram IP connectivity preference.
	DefaultPreferIP = "prefer-ipv6"

	// SecretKeyLength defines a length of the secret bytes used by Telegram and a
	// proxy.
	SecretKeyLength = 16

	// ConnectionIDBytesLength defines a count of random bytes used to generate a
	// stream/connection ids.
	ConnectionIDBytesLength = 16

	// TCPRelayReadTimeout defines a max time period between two consecuitive
	// reads from Telegram after which connection will be terminated. This is
	// required to abort stale connections.
	TCPRelayReadTimeout = 20 * time.Second
)

Variables

View Source
var (
	// ErrSecretEmpty is returned if you are trying to create a proxy but do not
	// provide a secret.
	ErrSecretEmpty = errors.New("secret is empty")

	// ErrSecretInvalid is returned if you are trying to create a proxy but secret
	// value is invalid (no host or payload are zeroes).
	ErrSecretInvalid = errors.New("secret is invalid")

	// ErrNetworkIsNotDefined is returned if you are trying to create a proxy but
	// network value is undefined.
	ErrNetworkIsNotDefined = errors.New("network is not defined")

	// ErrAntiReplayCacheIsNotDefined is returned if you are trying to create a
	// proxy but anti replay cache value is undefined.
	ErrAntiReplayCacheIsNotDefined = errors.New("anti-replay cache is not defined")

	// ErrIPBlocklistIsNotDefined is returned if you are trying to create a proxy
	// but ip blocklist instance is not defined.
	ErrIPBlocklistIsNotDefined = errors.New("ip blocklist is not defined")

	// ErrIPAllowlistIsNotDefined is returned if you are trying to create a proxy
	// but ip allowlist instance is not defined.
	ErrIPAllowlistIsNotDefined = errors.New("ip allowlist is not defined")

	// ErrEventStreamIsNotDefined is returned if you are trying to create a proxy
	// but event stream instance is not defined.
	ErrEventStreamIsNotDefined = errors.New("event stream is not defined")

	// ErrLoggerIsNotDefined is returned if you are trying to create a proxy but
	// logger is not defined.
	ErrLoggerIsNotDefined = errors.New("logger is not defined")
)

Functions

This section is empty.

Types

type AntiReplayCache

type AntiReplayCache interface {
	// Seen before checks if this set of bytes was observed before or not. If it
	// is required to store this information somewhere else, then it has to do
	// that.
	SeenBefore(data []byte) bool
}

AntiReplayCache is an interface that is used to detect replay attacks based on some traffic fingerprints.

Replay attacks are probe attacks whose main goal is to identify if server software can be classified in some way. For example, if you send some HTTP request to a web server, then you can expect that this server will respond with HTTP response back.

There is a problem though. Let's imagine, that connection is encrypted. Let's imagine, that it is encrypted with some static key like ShadowSocks. In that case, in theory, if you repeat the same bytes, you can get the same responses. Let's imagine, that you've cracked the key. then if you send the same bytes, you can decrypt a response and see its structure. Based on its structure you can identify if this server is SOCKS5, MTPROTO proxy etc.

This is just one example, maybe not the best or not the most relevant. In real life, different organizations use such replay attacks to perform some reverse engineering of the proxy, do some statical analysis to identify server software.

There are many ways how to protect your proxy against them. One is domain fronting which is a core part of mtg. Another one is to collect some 'handshake fingerprints' and forbid duplication.

So, it one is sending the same byte flow right after you (or a couple of hours after), mtg should detect that and reject this connection (or redirect to fronting domain).

type Event

type Event interface {
	// StreamID returns an identifier of the stream, connection, request, you name
	// it. All events within the same stream returns the same stream id.
	StreamID() string

	// Timestamp returns a timestamp when this event was generated.
	Timestamp() time.Time
}

Event is a data structure which is populated during mtg request processing lifecycle. Each request popluates many events:

  1. Client connected
  2. Request is finished
  3. Connection to Telegram server is established

and so on. All these events are data structures but all of them must conform the same interface.

type EventConcurrencyLimited

type EventConcurrencyLimited struct {
	// contains filtered or unexported fields
}

EventConcurrencyLimited is emitted when connection was declined because of the concurrency limit of the worker pool.

func NewEventConcurrencyLimited

func NewEventConcurrencyLimited() EventConcurrencyLimited

NewEventConcurrencyLimited creates a new EventConcurrencyLimited event.

func (EventConcurrencyLimited) StreamID

func (e EventConcurrencyLimited) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventConcurrencyLimited) Timestamp

func (e EventConcurrencyLimited) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventConnectedToDC

type EventConnectedToDC struct {

	// RemoteIP is an IP address of the Telegram server proxy has been connected
	// to.
	RemoteIP net.IP

	// DC is an index of the datacenter proxy has been connected to.
	DC int
	// contains filtered or unexported fields
}

EventConnectedToDC is emitted when mtg proxy has connected to a Telegram server.

func NewEventConnectedToDC

func NewEventConnectedToDC(streamID string, remoteIP net.IP, dc int) EventConnectedToDC

NewEventConnectedToDC creates a new EventConnectedToDC event.

func (EventConnectedToDC) StreamID

func (e EventConnectedToDC) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventConnectedToDC) Timestamp

func (e EventConnectedToDC) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventDomainFronting

type EventDomainFronting struct {
	// contains filtered or unexported fields
}

EventDomainFronting is emitted when we connect to a front domain instead of Telegram server.

func NewEventDomainFronting

func NewEventDomainFronting(streamID string) EventDomainFronting

NewEventDomainFronting creates a new EventDomainFronting event.

func (EventDomainFronting) StreamID

func (e EventDomainFronting) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventDomainFronting) Timestamp

func (e EventDomainFronting) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventFinish

type EventFinish struct {
	// contains filtered or unexported fields
}

EventFinish is emitted when we stop to manage a connection.

func NewEventFinish

func NewEventFinish(streamID string) EventFinish

NewEventFinish creates a new EventFinish event.

func (EventFinish) StreamID

func (e EventFinish) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventFinish) Timestamp

func (e EventFinish) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventIPBlocklisted

type EventIPBlocklisted struct {
	RemoteIP    net.IP
	IsBlockList bool
	// contains filtered or unexported fields
}

EventIPBlocklisted is emitted when connection was declined because IP address was found in IP blocklist.

func NewEventIPAllowlisted added in v2.1.6

func NewEventIPAllowlisted(remoteIP net.IP) EventIPBlocklisted

NewEventIPAllowlisted creates a NewEventIPBlocklisted event with a mark that it is supposed to be for allow list.

func NewEventIPBlocklisted

func NewEventIPBlocklisted(remoteIP net.IP) EventIPBlocklisted

NewEventIPBlocklisted creates a new EventIPBlocklisted event.

func (EventIPBlocklisted) StreamID

func (e EventIPBlocklisted) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventIPBlocklisted) Timestamp

func (e EventIPBlocklisted) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventIPListSize added in v2.1.5

type EventIPListSize struct {
	Size        int
	IsBlockList bool
	// contains filtered or unexported fields
}

EventIPListSize is emitted when mtg updates a contents of the ip lists: allowlist or blocklist.

func NewEventIPListSize added in v2.1.5

func NewEventIPListSize(size int, isBlockList bool) EventIPListSize

NewEventIPListSize creates a new EventIPListSize event.

func (EventIPListSize) StreamID added in v2.1.5

func (e EventIPListSize) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventIPListSize) Timestamp added in v2.1.5

func (e EventIPListSize) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventReplayAttack

type EventReplayAttack struct {
	// contains filtered or unexported fields
}

EventReplayAttack is emitted when mtg detects a replay attack on a connection.

func NewEventReplayAttack

func NewEventReplayAttack(streamID string) EventReplayAttack

NewEventReplayAttack creates a new EventReplayAttack event.

func (EventReplayAttack) StreamID

func (e EventReplayAttack) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventReplayAttack) Timestamp

func (e EventReplayAttack) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventStart

type EventStart struct {

	// RemoteIP is an IP address of the client.
	RemoteIP net.IP
	// contains filtered or unexported fields
}

EventStart is emitted when mtg proxy starts to process a new connection.

func NewEventStart

func NewEventStart(streamID string, remoteIP net.IP) EventStart

NewEventStart creates a new EventStart event.

func (EventStart) StreamID

func (e EventStart) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventStart) Timestamp

func (e EventStart) Timestamp() time.Time

Timestamp return a time when this event was generated.

type EventStream

type EventStream interface {
	// Send delivers an event to observers. Given context has to be respected. If
	// the context is closed, all blocking operations should be released ASAP.
	//
	// It is possible that context is closed but the message is delivered.
	// EventStream implementations should solve this issue somehow.
	Send(context.Context, Event)
}

EventStream is an abstraction that accepts a set of events produced by mtg. Its main goal is to inject your logging or monitoring system.

The idea is simple. When mtg works, it emits a set of events during a lifecycle of the requestor: EventStart, EventFinish etc. mtg is a producer which puts these events into a stream. Responsibility of the stream is to deliver this event to consumers/observers. There might be many different observers (for example, you want to have both statsd and prometheus), mtg should know nothing about them.

type EventTraffic

type EventTraffic struct {

	// Traffic is a count of bytes which were transmitted.
	Traffic uint

	// IsRead defines if we _read_ or _write_ to connection. A rule of thumb is
	// simple: EventTraffic is bound to a remote connection. Not to a client one,
	// but either to Telegram or front domain one.
	//
	// In the case of Telegram, isRead means that we've fetched some bytes from
	// Telegram to send it to a client.
	//
	// In the case of the front domain, it means that we've fetched some bytes
	// from this domain to send it to a client.
	IsRead bool
	// contains filtered or unexported fields
}

EventTraffic is emitted when we read/write some bytes on a connection.

func NewEventTraffic

func NewEventTraffic(streamID string, traffic uint, isRead bool) EventTraffic

NewEventTraffic creates a new EventTraffic event.

func (EventTraffic) StreamID

func (e EventTraffic) StreamID() string

StreamID returns a ID of the stream this event belongs to.

func (EventTraffic) Timestamp

func (e EventTraffic) Timestamp() time.Time

Timestamp return a time when this event was generated.

type IPBlocklist

type IPBlocklist interface {
	// Contains checks if given IP address belongs to this blocklist If. it is, a
	// connection is terminated .
	Contains(net.IP) bool

	// Run starts a background update procedure for a blocklist
	Run(time.Duration)

	// Shutdown stops a blocklist. It is assumed that none will access it after.
	Shutdown()
}

IPBlocklist filters requests based on IP address.

If this filter has an IP address, then mtg closes a request without reading anything from a socket. It also does not give such request to a worker pool, so in worst cases you can expect that you invoke this object more frequent than defined proxy concurrency.

type Logger

type Logger interface {
	// Named returns a new logger with a bound name. Name chaining is allowed and
	// appreciated.
	Named(name string) Logger

	// BindInt binds new integer parameter to a new logger instance.
	BindInt(name string, value int) Logger

	// BindStr binds new string parameter to a new logger instance.
	BindStr(name, value string) Logger

	// BindJSON binds a new JSON-encoded string to a new logger instance.
	BindJSON(name, value string) Logger

	// Printf is to support log.Logger behavior.
	Printf(format string, args ...interface{})

	// Info puts a message about some normal situation.
	Info(msg string)

	// InfoError puts a message about some normal situation but this situation is
	// related to a given error.
	InfoError(msg string, err error)

	// Warning puts a message about some extraordinary situation worth to look at.
	Warning(msg string)

	// WarningError puts a message about some extraordinary situation worth to
	// look at. This situation is related to a given error.
	WarningError(msg string, err error)

	// Debug puts a message useful for debugging only.
	Debug(msg string)

	// Debug puts a message useful for debugging only. This message is related to
	// a given error.
	DebugError(msg string, err error)
}

Logger defines an interface of the logger used by mtglib.

Each logger has a name. It is possible to stack names to organize poor-man namespaces. Also, each logger must be able to bind parameters to avoid pushing them all the time.

Example

logger := SomeLogger{} logger = logger.BindStr("ip", net.IP{127, 0, 0, 1})
logger.Info("Hello")

In that case, ip is bound as a parameter. It is a great idea to put this parameter somewhere in a log message.

logger1 = logger.BindStr("param1", "11") logger2 = logger.BindInt("param2",
11)

logger1 should see no param2 and vice versa, logger2 should not see param1 If you attach a parameter to a logger, parents should not know about that.

type Network

type Network interface {
	// Dial establishes context-free TCP connections.
	Dial(network, address string) (essentials.Conn, error)

	// DialContext dials using a context. This is a preferrable way of
	// establishing TCP connections.
	DialContext(ctx context.Context, network, address string) (essentials.Conn, error)

	// MakeHTTPClient build an HTTP client with given dial function. If nothing is
	// provided, then DialContext of this interface is going to be used.
	MakeHTTPClient(func(ctx context.Context, network, address string) (essentials.Conn, error)) *http.Client
}

Network defines a knowledge how to work with a network. It may sound fun but it encapsulates all the knowledge how to properly establish connections to remote hosts and configure HTTP clients.

For example, if you want to use SOCKS5 proxy, you probably want to have all traffic routed to this proxy: telegram connections, http requests and so on. This knowledge is encapsulated into instances of such interface.

mtglib uses Network for:

  1. Dialing to Telegram
  2. Dialing to front domain
  3. Doing HTTP requests (for example, for FireHOL ipblocklist).

type Proxy

type Proxy struct {
	// contains filtered or unexported fields
}

Proxy is an MTPROTO proxy structure.

func NewProxy

func NewProxy(opts ProxyOpts) (*Proxy, error)

NewProxy makes a new proxy instance.

func (*Proxy) DomainFrontingAddress

func (p *Proxy) DomainFrontingAddress() string

DomainFrontingAddress returns a host:port pair for a fronting domain.

func (*Proxy) Serve

func (p *Proxy) Serve(listener net.Listener) error

Serve starts a proxy on a given listener.

func (*Proxy) ServeConn

func (p *Proxy) ServeConn(conn essentials.Conn)

ServeConn serves a connection. We do not check IP blocklist and concurrency limit here.

func (*Proxy) Shutdown

func (p *Proxy) Shutdown()

Shutdown 'gracefully' shutdowns all connections. Please remember that it does not close an underlying listener.

type ProxyOpts

type ProxyOpts struct {
	// Secret defines a secret which should be used by a proxy.
	//
	// This is a mandatory setting.
	Secret Secret

	// Network defines a network instance which should be used for all network
	// communications made by proxies.
	//
	// This is a mandatory setting.
	Network Network

	// AntiReplayCache defines an instance of antireplay cache.
	//
	// This is a mandatory setting.
	AntiReplayCache AntiReplayCache

	// IPBlocklist defines an instance of IP blocklist.
	//
	// This is a mandatory setting.
	IPBlocklist IPBlocklist

	// IPAllowlist defines a whitelist of IPs to allow to use proxy.
	//
	// This is an optional setting, ignored by default (no restrictions).
	IPAllowlist IPBlocklist

	// EventStream defines an instance of event stream.
	//
	// This ia a mandatory setting.
	EventStream EventStream

	// Logger defines an instance of the logger.
	//
	// This is a mandatory setting.
	Logger Logger

	// BufferSize is a size of the copy buffer in bytes.
	//
	// Please remember that we multiply this number in 2, because when we relay
	// between proxies, we have to create 2 intermediate buffers: to and from.
	//
	// This is an optional setting.
	//
	// Deprecated: this setting is no longer makes any effect.
	BufferSize uint

	// Concurrency is a size of the worker pool for connection management.
	//
	// If we have more connections than this number, they are going to be
	// rejected.
	//
	// This is an optional setting.
	Concurrency uint

	// IdleTimeout is a timeout for relay when we have to break a stream.
	//
	// This is a timeout for any activity. So, if we have any message which will
	// pass to either direction, a timer is reset. If we have no any reads or
	// writes for this timeout, a connection will be aborted.
	//
	// This is an optional setting.
	IdleTimeout time.Duration

	// TolerateTimeSkewness is a time boundary that defines a time range where
	// faketls timestamp is acceptable.
	//
	// This means that if if you got a timestamp X, now is Y, then if |X-Y| <
	// TolerateTimeSkewness, then you accept a packet.
	//
	// This is an optional setting.
	TolerateTimeSkewness time.Duration

	// PreferIP defines an IP connectivity preference. Valid values are:
	// 'prefer-ipv4', 'prefer-ipv6', 'only-ipv4', 'only-ipv6'.
	//
	// This is an optional setting.
	PreferIP string

	// DomainFrontingPort is a port we use to connect to a fronting domain.
	//
	// This is required because secret does not specify a port. It specifies a
	// hostname only.
	//
	// This is an optional setting.
	DomainFrontingPort uint

	// AllowFallbackOnUnknownDC defines how proxy behaves if unknown DC was
	// requested. If this setting is set to false, then such connection will be
	// rejected. Otherwise, proxy will chose any DC.
	//
	// Telegram is designed in a way that any DC can serve any request, the
	// problem is a latency.
	//
	// This is an optional setting.
	AllowFallbackOnUnknownDC bool

	// UseTestDCs defines if we have to connect to production or to staging DCs of
	// Telegram.
	//
	// This is required if you use mtglib as an integration library for your
	// Telegram-related projects.
	//
	// This is an optional setting.
	UseTestDCs bool
}

ProxyOpts is a structure with settings to mtg proxy.

This is not required per se, but this is to shorten function signature and give an ability to conveniently provide default values.

type Secret

type Secret struct {
	// Key is a set of bytes used for traffic authentication.
	Key [SecretKeyLength]byte

	// Host is a domain fronting hostname.
	Host string
}

Secret is a data structure that presents a secret.

Telegram secret is not a simple string like "ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d". Actually, this is a serialized datastructure of 2 parts: key and host.

ee367a189aee18fa31c190054efd4a8e9573746f726167652e676f6f676c65617069732e636f6d
|-|-------------------------------|-------------------------------------------
p key                             hostname

Serialized secret starts with 'ee'. Actually, in the past we also had 'dd' secrets and prefixless ones. But this is history. Currently, we do have only 'ee' secrets which mean faketls + protection from statistical attacks on a length. 'ee' is a byte 238 (0xee).

After that, we have 16 bytes of the key. This is a random generated secret data of the proxy and this data is used to derive authentication schemas. These secrets are mixed into hmacs and sha256 checksums which are used to build AEAD ciphers for obfuscated2 protocol and ensure faketls handshake.

Host is a domain fronting hostname in latin1 (ASCII) encoding. This hostname should be used for SNI in faketls and MTG verifies it. Also, this is when mtg gets about a domain fronting hostname.

Secrets can be serialized into 2 forms: hex and base64. If you decode both forms into bytes, you'll get the same byte array. Telegram clients nowadays accept all forms.

func GenerateSecret

func GenerateSecret(hostname string) Secret

GenerateSecret makes a new secret with a given hostname.

func ParseSecret

func ParseSecret(secret string) (Secret, error)

ParseSecret parses a secret (both hex and base64 forms).

func (Secret) Base64

func (s Secret) Base64() string

Base64 returns a base64-encoded form of this secret.

func (Secret) Hex

func (s Secret) Hex() string

Hex returns a hex-encoded form of this secret (ee-secret).

func (Secret) MarshalText

func (s Secret) MarshalText() ([]byte, error)

MarshalText is to support text.Marshaller interface.

func (*Secret) Set added in v2.1.0

func (s *Secret) Set(text string) error

func (Secret) String

func (s Secret) String() string

String is to support fmt.Stringer interface.

func (*Secret) UnmarshalText

func (s *Secret) UnmarshalText(data []byte) error

UnmarshalText is to support text.Unmarshaller interface.

func (Secret) Valid

func (s Secret) Valid() bool

Valid checks if this secret is valid and can be used in proxy.

Directories

Path Synopsis
internal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL