README
¶
UDS - Packages for using Unix Domain Sockets
Usage Note
This package is considered beta. Until this note is removed, changes to this that cause an incompatibility will be noted with a Minor version change, not a Major version change in the overall repo.
TLDR
- Provides convience over standard net.Listen("unix") and net.Dial("unix")
- Provides high level packages to do streams and RPCs(protocol buffers/JSON)
- Provides authentication on the server
- Benchmarked against gRPC, mostly faster(except at 1kiB or lower) and WAY LESS allocations
- Linux/OSX only
Introduction
Go provides unix sockets via its net.Dial() and net.Listen() calls. But how to use these to achive IPC (interprocess communication) isn't straightforward.
This package and its sub-packages provide authenticated streaming and RPCs utilizing:
- Data chunks (whatever you want to send in []byte)
- JSON messages
- Protocol buffer(proto) messages
If you simply want to use UDS with an io.ReadWriteCloser interface without dealing with all the Dial()/Listen() noise (plus you get authentication), the main uds/ package is for you.
Works on Linux and OSX only. Haven't investigated all the BSD syscall options needed and don't want to tackle Windows (but I do take PRs).
Use cases
You need to do IPC for something cross platform (or at least Linux and OSX) and want that communication authenticated.
UDS is on most platforms (though not all options are). However, it is not the fastest IPC.
Shared memory seems to be the fasted (but its a pain). Various message queuing destroys UDS in performance, but they aren't very cross platform (so you'd have to use a different one per platform).
I've used UDS to communicate to sub-processes when I want to mount and unmount "plugins". Sometimes between languages.
Alternatives
There are some other packages out there, but I haven't seen them supporting auth.
gRPC works great for IPC on UDS, at least with protos. While I love gRPC, I wanted more options, authentication and much lower heap allocations.
Everyone loves some benchmarks
So I certainly haven't benched everything. But I did bench the proto RPC, as it uses the chunking rpc package, which uses chunked streams.
I compared this to gRPC as it provides a managed RPC mechansim over unix sockets. Interestingly, gRPC seem to handle the smallest data requests (1.0 kB) better than this package, but this seems to be do better for all others. I think the small data size test is due to my use of memory pools for reuse and slightly more overhead in my packets.
No matter what my internal settings, I would beat gRPC at 10kB and double their performance in the 102 kB size. To get better performance in large sizes, I had to add some kernel buffer space over the defaults, which lead to close to double performance.
But the real killer here is allocations. This decimates gRPC in heap allocation reduction. Keep this in mind for high performance applications. If you delve deep into making Go fast, everything points at one thing: keeping your allocations down. Once you leave the micro-benchmark world (like these benchmarks), your app starts crawling if the GC has to deal with lots of objects. The key to that is buffer reuse and gRPC's design for ease of use hurts that ability.
Platform was OSX running on an 8-core Macbook Pro, Circa 2019. You can guess that your Threadripper Linux box will do much better.
Test Results(uds):
==========================================================================
[Speed]
[16 Users][10000 Requests][1.0 kB Bytes] - min 65.286µs/sec, max 5.830322ms/sec, avg 373.437µs/sec, rps 42534.71
[16 Users][10000 Requests][10 kB Bytes] - min 108.214µs/sec, max 7.875121ms/sec, avg 576.661µs/sec, rps 27596.17
[16 Users][10000 Requests][102 kB Bytes] - min 760.896µs/sec, max 13.829294ms/sec, avg 1.188321ms/sec, rps 13419.44
[16 Users][10000 Requests][1.0 MB Bytes] - min 7.26139ms/sec, max 80.21521ms/sec, avg 14.509374ms/sec, rps 1101.91
[Allocs]
[10000 Requests][1.0 kB Bytes] - allocs 290,365
[10000 Requests][10 kB Bytes] - allocs 300,598
[10000 Requests][102 kB Bytes] - allocs 302,142
[10000 Requests][1.0 MB Bytes] - allocs 309,110
Test Results(grpc):
==========================================================================
[Speed]
[16 Users][10000 Requests][1.0 kB Bytes] - min 59.624µs/sec, max 3.571806ms/sec, avg 305.171µs/sec, rps 51137.15
[16 Users][10000 Requests][10 kB Bytes] - min 93.19µs/sec, max 2.397846ms/sec, avg 875.864µs/sec, rps 18216.72
[16 Users][10000 Requests][102 kB Bytes] - min 1.221068ms/sec, max 8.495421ms/sec, avg 4.434272ms/sec, rps 3606.63
[16 Users][10000 Requests][1.0 MB Bytes] - min 21.448849ms/sec, max 54.920306ms/sec, avg 34.307985ms/sec, rps 466.28
[Allocs]
[10000 Requests][1.0 kB Bytes] - allocs 1,505,165
[10000 Requests][10 kB Bytes] - allocs 1,681,061
[10000 Requests][102 kB Bytes] - allocs 1,947,529
[10000 Requests][1.0 MB Bytes] - allocs 3,250,309
Benchmark Guide:
[# Users] = number of simultaneous clients
[# Requests] = the total number of requests we sent
[# Bytes] = the size of our input and output requests
min = the minimum seen RTT
max = the maximum seen RTT
av = the average seen RTT
rps = requests per second
Documentation
¶
Overview ¶
Package uds provides a server and client for Unix Domain Sockets. This provides a lot of convenience around the "net" package for handling all the file setup and detecting closed connections. It also provides the ability to authenticate connections.
The package currently only works for Linux/Darwin, as those are the systems I use.
This package takes the stance that Read() and Write() calls by default should infinitely block unless the socket is closed. This eases development.
We also handle write only connections where Write() calls may not detect a closed connection. This can be done setting the WriteOnly option.
This package is fairly straight forward in that you can uses server.Conn objects and Client objects as io.ReadWriteClose objects. We also provide higher level clients that handle chunk data, a chunk data RPC client/server, json streams, json RPC client/server, ..
Unix/Linux Note:
Socket paths may have a length limit that is different than the normal filesystem. On OSX, you can receive "bind: invalid argument" when the name is too long. On Linux there seems to be an 108 character limit for socket path names. On OSX it is 104 character limit. https://github.com/golang/go/issues/6895 . I have set this as the default limit for all clients so I don't have to figure out the limit on every type of system and interpret non-sensical errors.
Index ¶
- type Client
- func (c *Client) Close() error
- func (c *Client) Read(b []byte) (int, error)
- func (c *Client) ReadByte() (byte, error)
- func (c *Client) ReadDeadline(t time.Time)
- func (c *Client) ReadTimeout(timeout time.Duration)
- func (c *Client) UnixConn() *net.UnixConn
- func (c *Client) Write(b []byte) (int, error)
- func (c *Client) WriteDeadline(t time.Time)
- func (c *Client) WriteOnly()
- func (c *Client) WriteTimeout(timeout time.Duration)
- type Conn
- func (c *Conn) Close() error
- func (c *Conn) Read(b []byte) (int, error)
- func (c *Conn) ReadByte() (byte, error)
- func (c *Conn) ReadDeadline(t time.Time)
- func (c *Conn) ReadTimeout(timeout time.Duration)
- func (c *Conn) UnixConn() *net.UnixConn
- func (c *Conn) Write(b []byte) (int, error)
- func (c *Conn) WriteDeadline(t time.Time)
- func (c *Conn) WriteOnly()
- func (c *Conn) WriteTimeout(timeout time.Duration)
- type Cred
- type ID
- type Server
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Client ¶
type Client struct {
// contains filtered or unexported fields
}
Client provides a UDS client for connecting to a UDS server.
func NewClient ¶
NewClient creates a new UDS client to the socket at socketAddr that must have the uid and gid specified. fileModes provides a list of acceptable file modes that the socket can be in (suggest 0770, 1770). If not specified, 0770 and 1770 are used.
func (*Client) Read ¶
Read implements io.Reader.Read(). This will block until it has read into the buffer. This differs from native behavior which times out. If you want to have read timeouts, use ReadDeadline()/ReadTimeout() to change. This must be done before every Read() call.
func (*Client) ReadDeadline ¶
ReadDeadline caused the next Read() call to timeout at t. Must be used before every Read() call that you want to have a timeout.
func (*Client) ReadTimeout ¶
ReadTimeout caused the next Read() call to timeout at time.Now().Add(timeout). Must be used before every Read() call that you want to have a timeout.
func (*Client) UnixConn ¶
UnixConn will return the underlying UnixConn object. You can use this to use its underlying methods or change buffering. Note that SetReadDeadline/SetWriteDeadline/SetDeadline do not work, use one of Client's methods instead.
func (*Client) Write ¶
Write implements io.Reader.Writer(). This will block until it has written the buffer. This differs from native behavior which times out. If you want to have write timeouts, use WriteDeadeline()/WriteTimeout() to change. This must be done before every Write() call.
func (*Client) WriteDeadline ¶
WriteDeadline caused the next Write() call to timeout at t. Must be used before every Write() call that you want to have a timeout.
func (*Client) WriteOnly ¶
func (c *Client) WriteOnly()
WriteOnly let's the Client know that it will only be used for writing. This will cause a *special* read to happen on all writes to make sure the connection isn't closed, as writes are not guaranteed to error on a UDS. If doing reads, this isn't required. If this is set and you read fro the Conn the read will panic.
func (*Client) WriteTimeout ¶
WriteTimeout caused the next Write() call to timeout at time.Now().Add(timeout). Must be used before every Write() call that you want to have a timeout.
type Conn ¶
type Conn struct {
Cred Cred
// contains filtered or unexported fields
}
Conn represents a UDS connection from a client. Must take a pointer if this will be copied after being received.
func (*Conn) Read ¶
Read implements io.Reader.Read(). This has an inifite read timeout. If you want to have a timeout, call ReadDeadline() or ReadTimeout() before calling. You must do this for every Read() call.
func (*Conn) ReadDeadline ¶
ReadDeadline caused the next Read() call to timeout at t. Must be used before every Read() call that you want to have a timeout.
func (*Conn) ReadTimeout ¶
ReadTimeout caused the next Read() call to timeout at time.Now().Add(timeout). Must be used before every Read() call that you want to have a timeout.
func (*Conn) UnixConn ¶
UnixConn will return the underlying UnixConn object. You can use this to use its underlying methods or change buffering. Note that .SetReadDeadline()/.SetWriteDeadline()/.SetDeadline() will not do anything. Use ReadTimeout()/ReadDeadline()/WriteTimeout()/WriteDeadline() defined on Conn.
func (*Conn) Write ¶
Write implements io.Writer.Write(). This has an inifite write timeout. If you want to have a timeout, call WriteDeadline() or WriteTimeout() before calling. You must do this for every Write() call.
func (*Conn) WriteDeadline ¶
WriteDeadline caused the next Write() call to timeout at t. Must be used before every Write() call that you want to have a timeout.
func (*Conn) WriteOnly ¶
func (c *Conn) WriteOnly()
WriteOnly let's the Conn know that this Conn will only be used for writing. This will cause a *special* read to happen on all writes to make sure the connection isn't closed, as writes are not guaranteed to error on a UDS. If doing reads, this isn't required. If this is set and you read from the Conn, Read() will panic.
func (*Conn) WriteTimeout ¶
WriteTimeout caused the next Write() call to timeout at time.Now().Add(timeout). Must be used before every Write() call that you want to have a timeout.
type Cred ¶
type Cred struct {
// PID is the process id of the process.
PID ID
// UID is the user id of the process.
UID ID
// GID is the group id of the process.
GID ID
}
Cred provides the credentials of the local process contacting the server.
type ID ¶
type ID int
ID represents a numeric ID. Go in various libraries stores IDs such as Uid or Gid as strings. However in other more OS specific libraries, it might be int or int32. This simply unifies that so it is easier to translate for whatever need you have. The string representation is probably a compatibility feature to allow an OS that doesn't use numeric IDs to work. However I am not interested in supporting those in this package.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server provides a Unix Domain Socket server that clients can connect on.
func NewServer ¶
NewServer creates a new UDS server that creates and listens to the file at socketPath. uid and gid are the uid and gid that file will be set to and fileMode is the file mode it will inherit. If sockerAddr exists this will attempt to delete it. Suggest fileMode of 0770.
func (*Server) Closed ¶
Closed returns a channel that returns an error when the connection to the server is closed. This can be because you have called Close(), the socket had a read error or the socket file was removed. An io.EOF error will not be returned (as this is normal operation). Normally this is used to block and return the final status of the server.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
example
|
|
|
highlevel
|
|
|
chunk
Package chunk provides a high level chunking client.
|
Package chunk provides a high level chunking client. |
|
chunk/rpc
Package rpc provides an RPC service for non-specific []byte in and []byte out data.
|
Package rpc provides an RPC service for non-specific []byte in and []byte out data. |
|
json/rpc
Package rpc provides a json RPC service.
|
Package rpc provides a json RPC service. |
|
json/stream
Package json provides a client that can be used on a *uds.Client or *uds.Conn to send streaming JSON values.
|
Package json provides a client that can be used on a *uds.Client or *uds.Conn to send streaming JSON values. |
|
proto/rpc
Package rpc provides a proto RPC service.
|
Package rpc provides a proto RPC service. |
|
proto/stream
Package stream provides a client that can be used on a *uds.Client or *uds.Conn to send streaming binary proto values.
|
Package stream provides a client that can be used on a *uds.Client or *uds.Conn to send streaming binary proto values. |