ngaro: github.com/db47h/ngaro/vm Index | Examples | Files

package vm

import "github.com/db47h/ngaro/vm"

Package vm provides an embeddable Ngaro Virtual Machine implementation.

Please visit http://forthworks.com/retro/ to get you started about the Retro language and the Ngaro Virtual Machine.

The main purpose of this implementation is to allow customization and communication between Retro programs and Go programs via custom opcodes and I/O handlers (i.e. scripting Go programs in Retro). The package examples demonstrate various use cases. For more details on I/O handling in the Ngaro VM, please refer to http://retroforth.org/docs/The_Ngaro_Virtual_Machine.html.

Another goal is to make the VM core as neutral as possible regarding the higher level language running on it. For example, the in-memory string encoding scheme is fully customizable. Retro specific behaviors are provided via the lang/retro package.

Custom opcodes are implemented by intercepting implicit calls to negative memory addresses. This allows the VM to be fully backwards compatible with existing Retro images while still providing enhanced capabilities. The maximum number of addressable cells is 2^31 when running in 32 bits mode (that's 8GiB or memory on the host). The range [-2^31 - 1, -1] is available for custom opcodes.

This implementation passes all tests from the retro-language test suite and its performance when running tests/core.rx is slightly better than with the reference implementations:

1.08s for this implementation, no custom opcodes, compiled with Go 1.7, linux/amd64
1.15s for the reference assembly implementation, linux/386
1.30s for the reference Go implementation, compiled with Go 1.7, linux/amd64
2.00s for the reference C implementation, compiled with gcc-5.4 -O3 -fomit-frame-pointer

For all intents and purposes, the VM behaves according to the specification. This is of particular importance to implementors of custom opcodes: the VM always increments the PC after each opcode, thus opcodes altering the PC must adjust it accordingly (i.e. set it to its real target minus one).

There's a caveat common to all Ngaro implementations: use of in, out and wait from the listener (the Retro interactive prompt) will not work as expected. This is because the listener will out/wait/in on the same ports as you do before you get a chance to read response values. This is of particular importance to users of custom IO handlers while testing: a value sitting in a control port can cause havok if not read and cleared in between two waits. To work around this issue, a synchronous OUT-WAIT-IN IO sequence must be compiled in a word, so that it will run atomically without interference from the listener. For example:

( io sends value n to port p, does a wait and puts response back on the stack.
  Note that the wait word does an `out 0 0` before issuing the real wait instruction )
: io ( np-n ) dup push out wait pop in ;

-1 5 io putn

should give you the total memory size.

Index

Examples

Package Files

core.go doc.go io.go io_helpers.go mem.go vm.go vm_int.go

Constants

const (
    CellBits = (1 << _log) << 3
)

Bits per Cell

func ClockLimiter Uses

func ClockLimiter(period, resolution time.Duration) (ticker func(i *Instance), ticks int64)

ClockLimiter returns a ticker function that sets the period between VM ticks. Its return values can be fed directly into Ticker().

A zero or negative period means no pause.

Since calling time.Sleep() on every tick is not efficient, the resolution sets the maximum real interval between calls to time.Sleep().

resolution is adjusted to be no smaller than period and so that the returned tick value is a power of two while keeping the period accurate.

Multiple ticker functions can be chained with a clock limiter:

// simulate a clock frequency of 20MHz with a call to the ticker function at most every 16ms (1/60s)
ticks, clkLimiter := ClockLimiter(time.Second/20e6, 16*time.Millisecond)

// wrap clkLimiter into a custom ticker
vm.Tick(ticks, func(i *vm.Instance) {
	// call the clock limiter
	clkLimiter(i)
	// update game engine
	game.Update(i)
})

func Save Uses

func Save(fileName string, mem []Cell, cellBits int) error

Save saves a Cell slice to an memory image file. The cellBits parameter specifies the number of bits per Cell in the file.

type Cell Uses

type Cell int

Cell is the basic type stored in a VM memory location.

const (
    OpNop Cell = iota
    OpLit
    OpDup
    OpDrop
    OpSwap
    OpPush
    OpPop
    OpLoop
    OpJump
    OpReturn
    OpGtJump
    OpLtJump
    OpNeJump
    OpEqJump
    OpFetch
    OpStore
    OpAdd
    OpSub
    OpMul
    OpDimod
    OpAnd
    OpOr
    OpXor
    OpShl
    OpShr
    OpZeroExit
    OpInc
    OpDec
    OpIn
    OpOut
    OpWait
)

Ngaro Virtual Machine Opcodes.

func Load Uses

func Load(fileName string, minSize, cellBits int) (mem []Cell, fileCells int, err error)

Load loads a memory image from file fileName. Returns a VM Cell slice ready to run from, the actual number of cells read from the file and any error. The cellBits parameter specifies the number of bits per Cell in the file.

type Codec Uses

type Codec interface {
    // Decode returns the decoded byte slice starting at position start in the specified slice.
    Decode(mem []Cell, start Cell) []byte
    // Encode writes the given byte slice at position start in specified slice.
    Encode(mem []Cell, start Cell, s []byte)
}

Codec encapsulates methods for encoding and decoding data stored in memory. This is primarily used to encode/decode strings: the VM needs to know how to encode/decode strings in some I/O operations, like when retrieving environment variables.

type InHandler Uses

type InHandler func(i *Instance, port Cell) error

InHandler is the function prototype for custom IN handlers.

type Instance Uses

type Instance struct {
    PC    int    // Program Counter (aka. Instruction Pointer)
    Mem   []Cell // Memory image
    Ports []Cell // I/O ports
    // contains filtered or unexported fields
}

Instance represents an Ngaro VM instance.

func New Uses

func New(mem []Cell, imageFile string, opts ...Option) (*Instance, error)

New creates a new Ngaro Virtual Machine instance.

The mem parameter is the Cell array used as memory image by the VM.

The imageFile parameter is the fileName that will be used to dump the contents of the memory image. It does not have to exist or even be writable as long as no user program requests an image dump.

Options will be set by calling SetOptions.

func (*Instance) Address Uses

func (i *Instance) Address() []Cell

Address returns the address stack. Note that value changes will be reflected in the instance's stack, but re-slicing will not affect it. To add/remove values on the address stack, use the Rpush and Rpop functions.

func (*Instance) Data Uses

func (i *Instance) Data() []Cell

Data returns the data stack. Note that value changes will be reflected in the instance's stack, but re-slicing will not affect it. To add/remove values on the data stack, use the Push and Pop functions.

func (*Instance) Depth Uses

func (i *Instance) Depth() int

Depth returns the data stack depth.

func (*Instance) Drop Uses

func (i *Instance) Drop()

Drop removes the top item from the data stack.

func (*Instance) Drop2 Uses

func (i *Instance) Drop2()

Drop2 removes the top two items from the data stack.

func (*Instance) In Uses

func (i *Instance) In(port Cell) error

In is the default IN handler for all ports.

func (*Instance) InstructionCount Uses

func (i *Instance) InstructionCount() int64

InstructionCount returns the number of instructions executed so far.

func (*Instance) Nos Uses

func (i *Instance) Nos() Cell

Nos returns the value of the Next item On the data Stack. Always returns 0 if Instance.Depth() is less than 2.

func (*Instance) Out Uses

func (i *Instance) Out(v, port Cell) error

Out is the default OUT handler for all ports.

func (*Instance) Pop Uses

func (i *Instance) Pop() Cell

Pop pops the value on top of the data stack and returns it.

func (*Instance) Push Uses

func (i *Instance) Push(v Cell)

Push pushes the argument on top of the data stack.

func (*Instance) PushInput Uses

func (i *Instance) PushInput(r io.Reader)

PushInput sets r as the current input io.Reader for the VM. When this reader reaches EOF, the previously pushed reader will be used.

func (*Instance) Rpop Uses

func (i *Instance) Rpop() Cell

Rpop pops the value on top of the address stack and returns it.

func (*Instance) Rpush Uses

func (i *Instance) Rpush(v Cell)

Rpush pushes the argument on top of the address stack.

func (*Instance) Run Uses

func (i *Instance) Run() (err error)

Run starts execution of the VM.

If an error occurs, the PC will will point to the instruction that triggered the error. The most likely error condition is an "index out of range" runtime.Error which can occur in the following cases:

- address or data stack full
- attempt to address memory outside of the range [0:len(i.Image)]
- use of a port number outside of the range [0:1024] in an I/O operation.

A full stack trace should be obtainable with:

fmt.Sprintf("%+v", err)

The VM will not error on stack underflows. i.e. drop always succeeds, and both Instance.Tos() and Instance.Nos() on an empty stack always return 0. This is a design choice that enables end users to use the VM interactively with Retro without crashes on stack underflows.

Please note that this behavior should not be used as a feature since it may change without notice in future releases.

If the VM was exited cleanly from a user program with the `bye` word, the PC will be equal to len(i.Image) and err will be nil.

Note that this package makes heavy use of the github.com/pkg/errors package. The "root cause" error can be obtained with errors.Cause().

If the last input stream gets closed, the VM will exit and the root cause error will be io.EOF. This is a normal exit condition in most use cases.

Shows how to load an image, setup the VM with multiple readers/init code.

Code:

imageFile := "testdata/retroImage"
// we know for a fact that this specific image file is 32 bits per Cell
img, _, err := vm.Load(imageFile, 50000, 32)
if err != nil {
    panic(err)
}

// output capture buffer
output := bytes.NewBuffer(nil)

// Setup the VM instance with os.Stdin as first reader, and we push another
// reader with some custom init code that will include and run the retro core tests.
i, err := vm.New(img, imageFile,
    vm.Input(os.Stdin),
    vm.StringCodec(retro.StringCodec),
    vm.Input(strings.NewReader("\"testdata/core.rx\" :include\n")),
    vm.Output(vm.NewVT100Terminal(output, nil, nil)))

// run it
if err == nil {
    err = i.Run()
}
// in interactive use, err may be io.EOF if any of the IO channels gets closed
// in which case this would be a normal exit condition
if err != nil && errors.Cause(err) != io.EOF {
    // Use %+v to get a nice stack trace
    fmt.Fprintf(os.Stderr, "%+v\n", err)
    return
}

// filter output to get the retro core test results.
b := bytes.Split(output.Bytes(), []byte{'\n'})
fmt.Printf("%s\n", b[len(b)-5])
fmt.Printf("%s\n", b[len(b)-4])

Output:

360 tests run: 360 passed, 0 failed.
186 words checked, 0 words unchecked, 37 i/o words ignored.

func (*Instance) SetOptions Uses

func (i *Instance) SetOptions(opts ...Option) error

SetOptions sets the provided options.

func (*Instance) SetTos Uses

func (i *Instance) SetTos(v Cell)

SetTos sets (changes) the value of the Top item On the data Stack. If the stack is empty, this function will be a no-op (i.e. Tos() will return 0).

func (*Instance) Tos Uses

func (i *Instance) Tos() Cell

Tos returns the value of the Top item On the data Stack. Always returns 0 if Instance.Depth() is 0.

func (*Instance) Wait Uses

func (i *Instance) Wait(v, port Cell) error

Wait is the default WAIT handler bound to ports 1, 2, 4, 5 and 8. It can be called manually by custom handlers that override default behaviour.

func (*Instance) WaitReply Uses

func (i *Instance) WaitReply(v, port Cell)

WaitReply writes the value v to the given port and sets port 0 to 1. This should only be used by WAIT port handlers.

type OpcodeHandler Uses

type OpcodeHandler func(i *Instance, opcode Cell) error

OpcodeHandler is the prototype for opcode handler functions. When an opcode handler is called, the VM's PC points to the opcode. Opcode handlers must take care of updating the VM's PC.

type Option Uses

type Option func(*Instance) error

An Option is a function for setting a VM Instance's options in New.

Note that Option functions directly set instance parameters without going through a delegate config structure, and since there is no locking mechanism to access an Instance's fields, Option functions must only be used in a call to New.

The only exception is from a ticker function registered with Ticker where the VM is actually paused during the call.

There are plans to change this and use a delegate config structure.

func AddressSize Uses

func AddressSize(size int) Option

AddressSize sets the address stack size. It will not erase the stack, and will panic if the requested size is not sufficient to hold the current stack. The default is 1024 cells.

func BindInHandler Uses

func BindInHandler(port Cell, handler InHandler) Option

BindInHandler binds the porvided IN handler to the given port.

The default IN handler behaves according to the specification: it reads the corresponding port value from Ports[port] and pushes it to the data stack. After reading, the value of Ports[port] is reset to 0.

Custom hamdlers do not strictly need to interract with Ports field. It is however recommended that they behave the same as the default.

func BindOpcodeHandler Uses

func BindOpcodeHandler(handler OpcodeHandler) Option

BindOpcodeHandler binds the given function to handle custom opcodes (i.e. opcodes with a negative value).

When an opcode handler is called, the VM's PC points to the opcode. Opcode handlers must take care of updating the VM's PC.

Demonstrates how to use custom opcodes. This example defines a custom opcode that pushes the n-th fibonacci number onto the stack.

Code:

fib := func(v vm.Cell) vm.Cell {
    var v0, v1 vm.Cell = 0, 1
    for v > 1 {
        v0, v1 = v1, v0+v1
        v--
    }
    return v1
}

handler := func(i *vm.Instance, opcode vm.Cell) error {
    switch opcode {
    case -1:
        i.SetTos(fib(i.Tos()))
        return nil
    default:
        return fmt.Errorf("Unsupported opcode value %d", opcode)
    }
}

img, err := asm.Assemble("test_fib_opcode", strings.NewReader(`
		.opcode fib -1	( define instruction fib as opcode -1 )
		46 fib
		`))
if err != nil {
    panic(err)
}

i, err := vm.New(img, "dummy", vm.BindOpcodeHandler(handler))
if err != nil {
    panic(err)
}

err = i.Run()
if err != nil {
    panic(err)
}

// So, what's Fib(46)?
fmt.Println(i.Data())

Output:

[1836311903]

func BindOutHandler Uses

func BindOutHandler(port Cell, handler OutHandler) Option

BindOutHandler binds the porvided OUT handler to the given port.

The default OUT handler just stores the given value in Ports[port]. A common use of OutHandler when using buffered I/O is to flush the output writer when anything is written to port 3. Such handler just ignores the written value, leaving Ports[3] as is.

Shows a common use of OUT port handlers.

Code:

imageFile := "testdata/retroImage"
img, _, err := vm.Load(imageFile, 50000, 32)
if err != nil {
    panic(err)
}

// Our out handler, will just take the written value, and return its square.
// The result will be stored in the bound port, so we can read it back with
// IN.
outputHandler := func(i *vm.Instance, v, port vm.Cell) error {
    i.Ports[port] = v * v
    return nil
}
// Create the VM instance with our port handler bound to port 42.
// We do not wire any output, we'll just read the results from the stack.
i, err := vm.New(img, imageFile,
    vm.Input(strings.NewReader(": square 42 out 42 in ; 7 square bye\n")),
    vm.BindOutHandler(42, outputHandler))
if err != nil {
    panic(err)
}

if err = i.Run(); err != nil && err != io.EOF {
    fmt.Fprintf(os.Stderr, "%v\n", err)
}

fmt.Println(i.Data())

Output:

[49]

func BindWaitHandler Uses

func BindWaitHandler(port Cell, handler WaitHandler) Option

BindWaitHandler binds the porvided WAIT handler to the given port.

WAIT handlers are called only if the value the following conditions are both true:

- the value of the bound I/O port is not 0
- the value of I/O port 0 is not 1

Upon completion, a WAIT handler should call the WaitReply method which will set the value of the bound port and set the value of port 0 to 1.

A simple WAIT handler that overrides the default implementation. It's used here to implement a (dummy) canvas. We'll need to override port 5 in order to report canvas availability and its size and implement the actual drawing on port 6. See http://retroforth.org/docs/The_Ngaro_Virtual_Machine.html

Code:

imageFile := "testdata/retroImage"
img, _, err := vm.Load(imageFile, 50000, 32)
if err != nil {
    panic(err)
}

waitHandler := func(i *vm.Instance, v, port vm.Cell) error {
    switch port {
    case 5: // override VM capabilities
        switch v {
        case -2:
            i.WaitReply(-1, port)
        case -3:
            i.WaitReply(1920, port)
        case -4:
            i.WaitReply(1080, port)
        default:
            // not a value that we handle ourselves, hand over the request
            // to the default implementation
            return i.Wait(v, port)
        }
        return nil
    case 6: // implement the canvas
        switch v {
        case 1:
            /* color */ _ = i.Pop()
        case 2:
            /* y */ _ = i.Pop()
            /* x */ _ = i.Pop()
            // draw a pixel at x, y...
        case 3:
            // more to implement...
        }
        // complete the request
        i.WaitReply(0, port)
    }
    return nil
}

// no output set as we don't care.
// out program first requests the VM size just to check that our override of
// port 5 properly hands over unknown requests to the default implementation.
i, err := vm.New(img, imageFile,
    vm.Input(strings.NewReader(
        ": cap ( n-n ) 5 out 0 0 out wait 5 in ;\n"+
            "-1 cap -2 cap -3 cap -4 cap bye\n")),
    vm.BindWaitHandler(5, waitHandler),
    vm.BindWaitHandler(6, waitHandler))
if err != nil {
    panic(err)
}

if err = i.Run(); err != nil && err != io.EOF {
    fmt.Fprintf(os.Stderr, "%+v\n", err)
}

fmt.Println(i.Data())

Output:

[50000 -1 1920 1080]

A more complex example of WAIT port handlers to communicate with Go. In this example, we use a pair of handlers: a request handler that will initiate a backround job, and a result handler to query and wait for the result.

Code:

imageFile := "testdata/retroImage"
img, _, err := vm.Load(imageFile, 50000, 32)
if err != nil {
    panic(err)
}

// we use a channel map to associate a task ID with a Go channel.
channels := make(map[vm.Cell]chan vm.Cell)

// our background task.
fib := func(v vm.Cell, c chan<- vm.Cell) {
    var v0, v1 vm.Cell = 0, 1
    for v > 1 {
        v0, v1 = v1, v0+v1
        v--
    }
    c <- v1
}

// The request hqndler will be bound to port 1000. Write 1 to this port with
// any arguments on the stack, then do a WAIT.
// It will respond on the same channel with a task ID.
execHandler := func(i *vm.Instance, v, port vm.Cell) error {
    // find an unused job id
    idx := vm.Cell(len(channels)) + 1
    // make i/o channel and save it
    c := make(chan vm.Cell)
    channels[idx] = c
    // launch job
    go fib(i.Pop(), c)
    // respond with channel ID
    i.WaitReply(idx, port)
    return nil
}

// The result handler will be wired to port 1001. Write the job id to this
// port followed by a wait and get the result with:
//
//	1001 IN
resultHandler := func(i *vm.Instance, v, port vm.Cell) error {
    c := channels[v]
    if c == nil {
        // no such channel. No need to error: if we do not reply, port 0
        // will be 0, so client code should check port 0 as well.
        return nil
    }
    i.WaitReply(<-c, port)
    delete(channels, v)
    return nil
}

// No output, we'll just grab the values from the stack on exit. Note that
// the port communication MUST be compiled in words (here fibGo and fibGet).
// Issuing IN/OUT/WAIT from the listener would fail because of interference
// from the I/O code.
i, err := vm.New(img, imageFile,
    vm.Input(strings.NewReader(
        `: fibGo ( n-ID ) 1 1000 out 0 0 out wait 1000 in ;
			 : fibGet ( ID-n ) 1001 out 0 0 out wait 1001 in ;
			 46 fibGo fibGet bye `)),
    vm.BindWaitHandler(1000, execHandler),
    vm.BindWaitHandler(1001, resultHandler))
if err != nil {
    panic(err)
}

if err = i.Run(); err != nil && err != io.EOF {
    fmt.Fprintf(os.Stderr, "%+v\n", err)
}

// So, what's Fib(46)?
fmt.Println(i.Data())

Output:

[1836311903]

func DataSize Uses

func DataSize(size int) Option

DataSize sets the data stack size. It will not erase the stack, and will panic if the requested size is not sufficient to hold the current stack. The default is 1024 cells.

func Input Uses

func Input(r io.Reader) Option

Input pushes the given io.Reader on top of the input stack.

func Output Uses

func Output(t Terminal) Option

Output configures the output Terminal. For simple I/O, the helper function NewVT100Terminal will build a Terminal wrapper around an io.Writer.

func SaveMemImage Uses

func SaveMemImage(fn func(filename string, mem []Cell) error) Option

SaveMemImage overrides the memory image dump function called when writing 1 to I/O port 4. The default is to call:

Save(i.imageFile, i.Mem, 0)

This is to allow saving images of different Cell sizes and to enable implementations of specific languages (like Retro) to do image shrinking based on some value in the VM instance's memory.

func StringCodec Uses

func StringCodec(e Codec) Option

StringCodec delegates string encoding/decoding in the memory image to the specified Codec. This is needed in file I/O where filenames are read from memory. Clients that make use of these I/O calls must configure a StringCodec. For Retro style encoding (one byte per Cell, 0 terminated), retro.StringCodec can be used as Codec. Implementations using othe encoding schemes, must provide their own Codec.

func Ticker Uses

func Ticker(fn func(i *Instance), ticks int64) Option

Ticker configures the VM to run the fn function every n VM ticks.

The ticks parameter is rounded up to the nearest power of two. If ticks <= 0, fn will never be called.

See ClockLimiter for an example use.

type OutHandler Uses

type OutHandler func(i *Instance, v, port Cell) error

OutHandler is the function prototype for custom OUT handlers.

type Terminal Uses

type Terminal interface {
    io.Writer
    Flush() error
    Size() (width int, height int)
    Clear()
    MoveCursor(x, y int)
    FgColor(fg int)
    BgColor(bg int)
    Port8Enabled() bool
}

Terminal encapsulates methods provided by a terminal output. All methods can be implemented as no-ops if the underlying output does not support the corresponding functionality.

Flush writes any buffered unwritten output.

Size returns the width and height of the terminal window. Should return 0, 0 if unsupported.

Clear clears the terminal window and moves the cursor to the top left.

MoveCursor moves the cursor to the specified column and row.

FgColor and BgColor respectively set the foreground and background color of all characters subsequently written.

Port8Enabled should return true if the MoveCursor, FgColor and BgColor methods have any effect.

func NewVT100Terminal Uses

func NewVT100Terminal(w io.Writer, flush func() error, size func() (width int, height int)) Terminal

NewVT100Terminal returns a new Terminal implementation that uses VT100 escape sequences to implement the Clear, CusrosrPos, FgColor and BgColor methods.

The caller only needs to provide the functions implementing Flush and Size. Either of these functions may be nil, in which case they will be implemented as no-ops.

type WaitHandler Uses

type WaitHandler func(i *Instance, v, port Cell) error

WaitHandler is the function prototype for custom WAIT handlers.

Package vm imports 9 packages (graph) and is imported by 3 packages. Updated 2018-06-03. Refresh now. Tools for package owners.