wire

Shiny Wire GUI

The shiny wire extends the Go shiny GUI library with the ability to run over a bidirectional communication layer, such as network connection, or on some systems a pipe or file (e.g. Plan 9). Defining the wire protocol also permits cross-programming language integrations so that client, server, or both can be written in other programming languages.

Using this library you can write your GUI application in precisely the same way as shiny. Use the special Main() function to wire it up to a server like this.

import (
	"golang.org/x/exp/shiny/screen"

	"supertxt.net/git/shiny-wire"
)

func main() {
	// Establish the connection with the server
	connreader := ...
        connwriter := ...

        wire.Main(connreader, connwriter, func(s screen.Screen) {
                w, err := s.NewWindow(&screen.NewWindowOptions{
                        Width:  800,
                        Height: 600,
                        Title:  "My App",
                })
		if err != nil {
			panic(err)
		}
		defer w.Release()

		select{}
	}

The client doesn't need access to a display, only a wire connection to a server that does. The server can be accessed over a network.

You can also set up a server using this library. Note that the server will need access to a display (linux/x11, macos/cocoa, windows) in the same way that a standard shiny application does.

import(
	"supertxt.net/git/shiny-wire"
)

func main() {
	// Wait for a connection from the client
	connreader := ...
	connwriter := ...

	wire.Server(connreader, connwriter)
}

You can try a simple demo drawing app yourself.

go run supertxt.net/git/shiny-wire/cmd/draw-wire@latest

Someday, there might be systems that provide a shiny device where the system provides a device to start a GUI session on-demand. If that happens then there is an even easier way to wire up a client using the library.

import (
        "golang.org/x/exp/shiny/screen"

        "supertxt.net/git/shiny-wire"
)

func main() {
        wire.MainDevShiny(func(s screen.Screen) {
                w, err := s.NewWindow(&screen.NewWindowOptions{
                        Width:  800,
                        Height: 600,
                        Title:  "My App",
                })
                if err != nil {
                        panic(err)
                }
                defer w.Release()

		select {}
        }

The Wire

All interactions are triggered by the client with messages in the following format.

len[uint32] type[byte] ...

The size of the overall client message is sent first, the type of the message, and then the message-specific payload comes after. The portion in braces indicates the data type and length of each piece packed using Big Endian following network byte ordering. Depending on the type of the message the server may respond with a payload (or not) that is specific to the type of the message.

Note: clients are expected to send their messages and receive their responses in a serial fashion without interleaving. This mirrors the way that shiny works with all UI interactions occurring on a single thread/goroutine synchronously.

Responses can only come directly after a client has triggered them and are in the following format if the message requires a response at all.

len[uint32]

Notice that no type is specified here, but type information can be communicated in certain cases, such as events. See the response message details in each message description below for more information.

Data Types

There are certain common data types used in messages. One is the wid (window identifier) used to identify the particular window in many of the window based messages, which is usually the first item in a message after the message type. It's represented as an unsigned 16-bit number like this with Big Endian byte order.

wid[uint16]

Similarly, there is a texture identifier (tid) that works exactly the same way.

tid[uint16]

Points (point) are a sequence of 32-bit integers, first X and then Y.

x[int32] y[int32] # from now on these are referred to as 'point'

Rectangles (rect) are a sequence of points, Min and then Max.

min[point] max[point]

Colors (color) are a sequence of 8-bit unsigned in the sequence of R, G, B, and then A.

r[uint8] g[uint8] b[uint8] a[uint8]

Draw Operation (drawop) is a Porter-Duff compositing operator. A value of 0 specifies "(src in mask) over dst." A value of 1 specifies "src in mask."

drawOp[int32] 

An affine transformation matrix (aff3) is in row major order, where the bottom row is implicitly [0 0 1]. m[3*r + c] is the element in the r'th row and c'th column.

aff3[float64[6]]

Events

The following events may be responded by WINDOW_NEXT_EVENT indicated by the first byte of the response from that message.

LIFECYCLE_EVENT byte = 1
SIZE_EVENT      byte = 2
PAINT_EVENT     byte = 3
KEY_EVENT       byte = 4
MOUSE_EVENT     byte = 5
TOUCH_EVENT     byte = 6

Lifecycle

A lifecycle event is a change from an old stage to a new stage. Stage is a stage in the app's lifecycle. The values are ordered, so that a lifecycle change from stage From to stage To implicitly crosses every stage in the range (min, max], exclusive on the low end and inclusive on the high end, where min is the minimum of From and To, and max is the maximum. Possible values for stage is Dead (0), Alive (1), Visible (2), Focused (3).

Lifecycle event response has the following form:

fromstage[uint32] tostage[uint32]

Dead is the zero stage. No lifecycle change crosses this stage, but:

• A positive change from this stage is the very first lifecycle change.
• A negative change to this stage is the very last lifecycle change.

Alive means that the app is alive.

• A positive cross means that the app has been created.
• A negative cross means that the app is being destroyed.

Visible means that the app window is visible.

• A positive cross means that the app window has become visible.
• A negative cross means that the app window has become invisible.

Focused means that the app window has the focus.

• A positive cross means that the app window has gained the focus.
• A negative cross means that the app window has lost the focus.

Size

Size holds the dimensions, physical resolution and orientation of the app's window. WidthPx and HeightPx are the window's dimensions in pixels. WidthPt and HeightPt are the window's physical dimensions in points (1/72 of an inch). The values are based on PixelsPerPt and are therefore approximate, as per the comment on PixelsPerPt. PixelsPerPt is the window's physical resolution. Orientation is the orientation of the device screen.

widthpx[int32] heightpx[int32] widthpt[float32] heightpt[float32] pixelsperpt[float32] orientation[int32]

OrientationUnknown(0) means device orientation cannot be determined. OrientationPortrait(1) is a device oriented so it is tall and thin. OrientationLandscape(2) is a device oriented so it is short and wide.

Paint

Paint event indicates that the app is ready to paint the next frame of the GUI. A frame is completed by sending the WINDOW_PUBLISH message.

external[byte]

External(1) is for paint events sent by the screen driver. An external event may be sent at any time in response to an operating system event, for example the window opened, was resized, or the screen memory was lost. Programs actively drawing to the screen as fast as vsync allows should ignore external paint events to avoid a backlog of paint events building up.

Key

rune[utf8-codepoint] code[uint32] modifiers[uint32] direction[uint8]

Rune is the meaning of the key event as determined by the operating system. The mapping is determined by system-dependent current layout, modifiers, lock-states, etc. If non-negative, it is a Unicode codepoint: pressing the 'a' key generates different Runes 'a' or 'A' (but the same Code) depending on the state of the shift key. If -1, the key does not generate a Unicode codepoint. To distinguish them, look at Code.

Code is the identity of the physical key relative to a notional "standard" keyboard, independent of current layout, modifiers, lock-states, etc. For standard key codes, its value matches USB HID key codes. Compare its value to uint32-typed constants in this package, such as CodeLeftShift and CodeEscape. Pressing the regular '2' key and number-pad '2' key (with Num-Lock) generate different Codes (but the same Rune).

Modifiers is a bitmask representing a set of modifier keys: ModShift(1), ModControl(2), ModAlt(4), ModMeta(8).

Direction is the direction of the key event: DirPress(1), DirRelease(2), or DirNone(0) (for key repeats).

Mouse

x[float32] y[float32] button[int32] modifiers[uint32] direction[uint8]

X and Y are the mouse location, in pixels.

Button is the mouse button being pressed or released. Its value may be zero, for a mouse move or drag without any button change.

• ButtonNone (0)
• ButtonLeft (1)
• ButtonMiddle (2)
• ButtonRight (3)
• ButtonWheelUp (-1)
• ButtonWheelDown (-2)
• ButtonWheelLeft (-3)
• ButtonWheelRight (-4)

Modifiers is a bitmask representing a set of modifier keys: ModShift(1), ModControl(2), ModAlt(4), ModMeta(8).

Direction is the direction of the mouse event: DirPress(1), DirRelease(2), or DirNone(0) (for mouse moves or drags).

Touch

x[float32] y[float32] sequence[int64] type[byte]

X and Y are the touch location, in pixels.

Sequence is the sequence number. The same number is shared by all events in a sequence. A sequence begins with a single TypeBegin, is followed by zero or more TypeMoves, and ends with a single TypeEnd. A Sequence distinguishes concurrent sequences but its value is subsequently reused.

Type is the touch type.

• TypeBegin(0) - a user first touching the device
• TypeMove(1) - user dragging across the device
• TypeEnd(2) - user no longer touching the device

Messages

Here is the list of the message types:

NEW_WINDOW          byte = 1
WINDOW_RELEASE      byte = 2
WINDOW_UPLOAD       byte = 3
WINDOW_FILL         byte = 4
WINDOW_PUBLISH      byte = 5
WINDOW_NEXT_EVENT   byte = 6
WINDOW_DRAW         byte = 7
WINDOW_DRAW_UNIFORM byte = 8
WINDOW_COPY         byte = 9
WINDOW_SCALE        byte = 10

NEW_TEXTURE     byte = 11
TEXTURE_RELEASE byte = 12
TEXTURE_SIZE    byte = 13
TEXTURE_BOUNDS  byte = 14
TEXTURE_UPLOAD  byte = 15
TEXTURE_FILL    byte = 16

New Window

Creates a new window for the screen.

wid width[uint16] height[uint16] title[n]

The window id is generated as a session unique number by the client for use in future window operations for that window. Width and Height specify the dimensions of the new window. If Width or Height are zero, a driver-dependent default will be used for each zero value dimension. Title specifies the window title.

error[n]

An error is a non-empty utf8 string. Otherwise, it is successful.

Window Release

Release closes the window.

wid

The window id is the identifier chosen by the client when creating the new window.

Window Upload

Upload uploads the sub-Buffer defined by src and sr to the destination (the method receiver), such that sr.Min in src-space aligns with dp in dst-space. The destination's contents are overwritten; the draw operator is implicitly draw.Src. When uploading to a Window, there will not be any visible effect until Publish is called.

wid dp[point] src[rect] stride[int32] rect[rect] pix[uint8 * n]

Stride is the Pix stride (in bytes) between vertically adjacent pixels. Rect is the image's bounds. Pix holds the image's pixels, in R, G, B, A order. The pixel at (x, y) starts at Pix[(y-Rect.Min.Y)*Stride + (x-Rect.Min.X)*4].

Window Fill

Fill fills that part of the destination (the method receiver) defined by dr with the given color. When filling a Window, there will not be any visible effect until Publish is called.

wid dr[rect] src[color] op[drawOp]

Window Publish

Publish flushes any pending Upload and Draw calls to the window, and swaps the back buffer to the front.

wid

The response message gives a publish result information.

backbufferpreserved[byte]

The backbufferpreserved is 0 for false, 1 for true.

Window Next Event

NextEvent returns the next event in the deque. It blocks until such an event has been sent.

wid

Responses are in the following form from the server.

eventtype[byte] eventpayload...

The event type values and the specific payloads are described above.

Window Draw

Draw draws the sub-Texture defined by src and sr to the destination (the method receiver). src2dst defines how to transform src coordinates to dst coordinates. For example, if src2dst is the matrix

m00 m01 m02 m10 m11 m12

then the src-space point (sx, sy) maps to the dst-space point (m00*sx + m01*sy + m02, m10*sx + m11*sy + m12).

wid src2dst[aff3] src[tid] sr[rect] op[drawOp]

Window Draw Uniform

DrawUniform is like Draw except that the src is a uniform color instead of a Texture.

wid src2dst[aff3] src[color] sr[rect] op[drawOp]

Window Copy

Copy copies the sub-Texture defined by src and sr to the destination (the method receiver), such that sr.Min in src-space aligns with dp in dst-space.

wid dp[point] src[tid] sr[rect] op[drawOp]

Window Scale

Scale scales the sub-Texture defined by src and sr to the destination (the method receiver), such that sr in src-space is mapped to dr in dst-space.

wid dr[rect] src[tid] sr[rect] op[drawOp]

New Texture

Creates a new texture for the screen.

tid sz[point]

The texture id is generated as a session unique number by the client for use in future texture operations. The sz provides the size of the texture.

error[n]

An error is a non-empty utf8 string. Otherwise, it is successful.

Texture Release

Release the texture from memory.

tid

The texture id is the identifier chosen by the client when creating the new texture.

Texture Size

Returns the size of the Texture's image.

tid

The response message provides the size.

size[point]

Texture Bounds

Bounds returns the bounds of the Texture's image. It is equal to Rectangle{Max: t.Size()}.

tid

The response message provides the bounds.

bounds[rect]

Texture Upload

Upload uploads the sub-Buffer defined by src and sr to the destination (the method receiver), such that sr.Min in src-space aligns with dp in dst-space. The destination's contents are overwritten; the draw operator is implicitly draw.Src.

tid dp[point] src[rect] stride[int32] rect[rect] pix[uint8 * n]

Stride is the Pix stride (in bytes) between vertically adjacent pixels. Rect is the image's bounds. Pix holds the image's pixels, in R, G, B, A order. The pixel at (x, y) starts at Pix[(y-Rect.Min.Y)*Stride + (x-Rect.Min.X)*4].

Texture Fill

Fill fills that part of the destination (the method receiver) defined by dr with the given color.

wid dr[rect] src[color] op[drawOp]