kernel

package
v0.10.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 14, 2024 License: MIT Imports: 28 Imported by: 0

Documentation

Overview

Package kernel handles the lower level communication with the Jupyter client.

It creates the sockets, encoding, validation, etc.

Reference documentation: https://jupyter-client.readthedocs.io/en/latest/messaging.html

Index

Constants

View Source
const (
	StatusStarting = "starting"
	StatusBusy     = "busy"
	StatusIdle     = "idle"
)
View Source
const (
	// StreamStdout defines the stream name for standard out on the front-end. It
	// is used in `PublishWriteStream` to specify the stream to write to.
	StreamStdout = "stdout"

	// StreamStderr defines the stream name for standard error on the front-end. It
	// is used in `PublishWriteStream` to specify the stream to write to.
	StreamStderr = "stderr"
)
View Source
const JupyterDataDirEnv = "JUPYTER_DATA_DIR"

JupyterDataDirEnv is the name of the environment variable pointing to the data files for Jupyter, including kernel configuration.

Variables

View Source
var CaptureSignals = []os.Signal{
	os.Interrupt, syscall.SIGHUP, syscall.SIGTERM,
}

CaptureSignals list all signals to be captured: except of `os.Interrupt` (Control+C), all others trigger a clean exit of GoNB kernel.

Notice `os.Interrupt` is used by Jupyter to signal to interrupt the execution of the current cell.

View Source
var (
	// ProtocolVersion defines the Jupyter protocol version. See differences here:
	// https://jupyter-client.readthedocs.io/en/stable/messaging.html#changelog
	//
	// Version >= 5.2 encodes cursor pos as one position per unicode character.
	// (https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets)
	// But still Jupyter-Lab is using UTF-16 encoding for cursor-pos.
	ProtocolVersion = "5.4"
)

Functions

func Install

func Install(extraArgs []string, forceDeps, forceCopy bool) error

Install gonb in users local Jupyter configuration, making it available. It assumes the kernel is implemented by the same binary calling this function (os.Args[0]) and that the flag to pass the `connection_file` is `--kernel`.

If the binary is under `/tmp` (or if forceCopy is true), it is copied to the location of the kernel configuration, and that copy is used.

If forceDeps is true, installation will succeed even with missing dependencies.

Documentation: https://jupyter-client.readthedocs.io/en/latest/kernels.html#kernelspecs

func JupyterCursorPosToBytePos

func JupyterCursorPosToBytePos(cellContent string, cursorPosUTF16 int) int

JupyterCursorPosToBytePos converts a `cursor_pos` in a cell, encoded in UTF-16, to a byte position in the cell text, encoded in UTF-8 by Go.

Documentation is a bit confusing, according to this: https://jupyter-client.readthedocs.io/en/stable/messaging.html#cursor-pos-and-unicode-offsets For version >= 5.2 cursor pos should be given as number of unicode runes (?). But in practice Jupyter is sending UTF-16

func JupyterToLinesAndCursor

func JupyterToLinesAndCursor(cellContent string, cursorPosUTF16 int) (lines []string, cursorLine, cursorCol int)

JupyterToLinesAndCursor takes as input the cell contents and a cursor (as a position in UTF16 points) and returns a slice with the individual lines and a cursor split into cursorLine and cursorCol, bytes based (meaning cursorCol is the position in the line in number of bytes).

func LogDisplayData

func LogDisplayData(data MIMEMap)

LogDisplayData prints out the display data using `klog`.

func NewJupyterStreamWriter

func NewJupyterStreamWriter(msg Message, stream string) io.Writer

NewJupyterStreamWriter returns an io.Writer that forwards what is written to the Jupyter client, under the given stream name.

func PublishData

func PublishData(msg Message, data Data) error

PublishData is a wrapper to either PublishExecuteResult or PublishDisplayData, depending if the parent message was "execute_result" or something else.

func PublishDisplayData

func PublishDisplayData(msg Message, data Data) error

PublishDisplayData publishes data of arbitrary data-types.

func PublishExecuteInput

func PublishExecuteInput(msg Message, code string) error

PublishExecuteInput publishes a status message notifying front-ends of what code is currently being executed.

func PublishExecuteResult

func PublishExecuteResult(msg Message, data Data) error

PublishExecuteResult publishes using "execute_result" method. Very similar to PublishDisplayData, but in response to an "execute_request" message.

func PublishExecutionError

func PublishExecutionError(msg Message, err string, trace []string, name string) error

PublishExecutionError publishes a serialized error that was encountered during execution.

func PublishHtml

func PublishHtml(msg Message, html string) error

PublishHtml is a shortcut to PublishData for HTML content.

func PublishJavascript

func PublishJavascript(msg Message, js string) error

PublishJavascript is a shortcut to PublishData for javascript content to be executed.

Note: `text/javascript` mime-type (protocol.MIMETextJavascript) is not supported by VSCode, it's displayed as text. So using this won't work in VSCode.

func PublishKernelStatus

func PublishKernelStatus(msg Message, status string) error

PublishKernelStatus publishes a status message notifying front-ends of the state the kernel is in. It supports the states "starting", "busy", and "idle".

func PublishMarkdown

func PublishMarkdown(msg Message, markdown string) error

PublishMarkdown is a shortcut to PublishData for markdown content.

func PublishUpdateDisplayData

func PublishUpdateDisplayData(msg Message, data Data) error

PublishUpdateDisplayData is like PublishDisplayData, but expects `transient.display_id` to be set. If the "display_id" is new, it will publish the data with the given "display_id" as usual, creating a new block ("<div>"). If it has already been seen, instead it updates the previously create display data on that "display_id".

func PublishWriteStream

func PublishWriteStream(msg Message, stream string, data string) error

PublishWriteStream prints the data string to a stream on the front-end. This is either `StreamStdout` or `StreamStderr`.

func SendKernelInfo

func SendKernelInfo(msg Message, version string) error

SendKernelInfo sends a kernel_info_reply message.

Types

type CommClose

type CommClose struct {
	// CommId is a UUID that identify this "comm" channel.
	CommId string `json:"comm_id"`

	// Data  key is always a dict and can be any extra JSON information used
	// in initialization of the comm.
	Data map[string]any `json:"data"`
}

CommClose is part of the "custom messages" protocol. The corresponding message type is "comm_close". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages

type CommInfoReply

type CommInfoReply struct {
	// Status should be set to 'ok' if the request succeeded or 'error',
	// or with error information as in all other replies.
	Status string `json:"status"`

	// Comms is a dictionary of comm_id (uuids) to a dictionary of fields.
	// The only field documented is "target_name".
	Comms map[string]map[string]string `json:"comms"`
}

CommInfoReply message sent in reply to a "comm_info_request". https://jupyter-client.readthedocs.io/en/latest/messaging.html#comm-info

type CommMsg

type CommMsg struct {
	// CommId is a UUID that identify this "comm" channel.
	CommId string `json:"comm_id"`

	// Data  key is always a dict and can be any extra JSON information used
	// in initialization of the comm.
	Data map[string]any `json:"data"`
}

CommMsg is part of the "custom messages" protocol. The corresponding message type is "comm_msg". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages

type CommOpen

type CommOpen struct {
	// CommId is a UUID that identify this new "comm" channel being opened (?).
	// Documentation is not clear how these should be used.
	CommId string `json:"comm_id"`

	// TargetName: documentation is not clear what it is, but it says:
	// - If the target_name key is not found on the receiving side,
	//   then it should immediately reply with a comm_close message to avoid an
	//   inconsistent state.
	TargetName string `json:"target_name"`

	// Data key is always a dict and can be any extra JSON information used
	// in initialization of the comm.
	Data map[string]any `json:"data"`
}

CommOpen is part of the "custom messages" protocol. The corresponding message type is "comm_open". https://jupyter-client.readthedocs.io/en/latest/messaging.html#custom-messages

type CompleteReply

type CompleteReply struct {
	Status      string   `json:"status"`
	Matches     []string `json:"matches"`
	CursorStart int      `json:"cursor_start"`
	CursorEnd   int      `json:"cursor_end"`
	Metadata    MIMEMap  `json:"metadata"`
}

CompleteReply message sent in reply to a "complete_request": used for auto-complete of the code under a certain position of the cursor.

type ComposedMsg

type ComposedMsg struct {
	Header       zmqMsgHeader
	ParentHeader zmqMsgHeader
	Metadata     map[string]any
	Content      any
}

ComposedMsg represents an entire message in a high-level structure.

func NewComposed

func NewComposed(msgType string, parent ComposedMsg) (*ComposedMsg, error)

NewComposed creates a new ComposedMsg to respond to a parent message. This includes setting up its headers.

type Data

type Data = struct {
	Data      MIMEMap
	Metadata  MIMEMap
	Transient MIMEMap
}

Data is the exact structure returned to Jupyter. It allows to fully specify how a value should be displayed.

type HelpLink struct {
	Text string `json:"text"`
	URL  string `json:"url"`
}

HelpLink stores data to be displayed in the help menu of the notebook.

type InspectReply

type InspectReply struct {
	Status   string  `json:"status"`
	Found    bool    `json:"found"`
	Data     MIMEMap `json:"data"`
	Metadata MIMEMap `json:"metadata"`
}

InspectReply message sent in reply to an "inspect_request": used for introspection on the code under a certain position of the cursor.

type InterruptFn added in v0.10.0

type InterruptFn func(id SubscriptionId)

InterruptFn is called on its own goroutine.

type InvalidSignatureError

type InvalidSignatureError struct{}

InvalidSignatureError is returned when the signature on a received message does not validate.

func (*InvalidSignatureError) Error

func (e *InvalidSignatureError) Error() string

type Kernel

type Kernel struct {

	// ExecCounter is incremented each time we run user code in the notebook.
	ExecCounter int

	// Interrupted indicates whether cell/shell currently being executed was Interrupted.
	Interrupted atomic.Bool

	// JupyterKernelId is a unique id associated to the kernel by Jupyter.
	// It's different from the id created by goexec.State to identify the temporary Go
	// code compilation.
	// The use so far, besides to match Jupyter logs, is also to create URL to websocket port
	// for the session (see `dispatcher/comm.go`).
	JupyterKernelId string

	// KnownBlockIds are display data blocks with a "display_id" that have already been created, and
	// hence should be updated (instead of created anew) in calls to PublishUpdate
	KnownBlockIds common.Set[string]
	// contains filtered or unexported fields
}

func New

func New(connectionFile string) (*Kernel, error)

New builds and start a kernel. Various goroutines are started to poll for incoming messages. It automatically handles the heartbeat.

Incoming messages should be listened to using Kernel.Stdin, Kernel.Shell and Kernel.Control.

The Kernel can be stopped by calling Kernel.Stop. And one can wait for the kernel clean up of the various goroutines, after it is stopped, by calling kernel.ExitWait.

The `connectionFile` is created by Jupyter with information on which ports to connect to each socket. The path itself is also used to extract the JupyterKernelId associated with this instance of the kernel.

func (*Kernel) CallInterruptSubscribers added in v0.10.1

func (k *Kernel) CallInterruptSubscribers()

CallInterruptSubscribers in a separate goroutine each. Meant to be called when JupyterServer sends a kernel interrupt (either a SIGINT, or a `interrupt_request` message to interrupt).

func (*Kernel) Control

func (k *Kernel) Control() <-chan Message

Control returns the reading channel where incoming control messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

func (*Kernel) ExitWait

func (k *Kernel) ExitWait()

ExitWait will wait for the kernel to be stopped and all polling goroutines to finish.

func (*Kernel) FromWireMsg

func (k *Kernel) FromWireMsg(zmqMsg zmq4.Msg) Message

FromWireMsg translates a multipart ZMQ messages received from a socket into a ComposedMsg struct and a slice of return identities. This includes verifying the message signature.

This "Wire Protocol" of the messages is described here: https://jupyter-client.readthedocs.io/en/latest/messaging.html#the-wire-protocol

func (*Kernel) HandleInterrupt

func (k *Kernel) HandleInterrupt()

HandleInterrupt will configure the kernel to listen to the system SIGINT, sent by default by Jupyter to indicate an interruption of the kernel (similar to a Control+C).

So instead of the kernel dying, it will recover, and where appropriate interrupt other subprocesses it may have spawned.

func (*Kernel) IsStopped

func (k *Kernel) IsStopped() bool

IsStopped returns whether the Kernel has been stopped.

func (*Kernel) Shell

func (k *Kernel) Shell() <-chan Message

Shell returns the reading channel where incoming shell messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

func (*Kernel) Stdin

func (k *Kernel) Stdin() <-chan Message

Stdin returns the reading channel where incoming stdin messages are received.

One should also select for the Kernel.StoppedChan, to check if connection to kernel was closed.

These are messages received from a console like input, while executing a command. For MessageImpl.PipeExecToJupyterWithInput to work, you need to call MessageImpl.DeliverInput() on these messages.

func (*Kernel) Stop

func (k *Kernel) Stop()

Stop the Kernel, indicating to all polling processes to quit.

func (*Kernel) StoppedChan

func (k *Kernel) StoppedChan() <-chan struct{}

StoppedChan returns a channel that can be listened (`select`) to check when the kernel is stopped. The channel will be closed when the Kernel is stopped.

func (*Kernel) SubscribeInterrupt added in v0.10.0

func (k *Kernel) SubscribeInterrupt(fn InterruptFn) SubscriptionId

SubscribeInterrupt registers `fn` to be called if any interruptions occur. It returns a SubscriptionId that needs to be used to unsubscribe to it later.

func (*Kernel) ToWireMsg

func (k *Kernel) ToWireMsg(c *ComposedMsg) ([][]byte, error)

ToWireMsg translates a ComposedMsg into a multipart ZMQ message ready to send, and signs it. This does not add the return identities or the delimiter.

func (*Kernel) UnsubscribeInterrupt added in v0.10.0

func (k *Kernel) UnsubscribeInterrupt(id SubscriptionId)

UnsubscribeInterrupt stops being called back in case of interruptions. It takes the `id` returned by [SubscribeInterrupt].

type KernelInfo

type KernelInfo struct {
	ProtocolVersion       string             `json:"protocol_version"`
	Implementation        string             `json:"implementation"`
	ImplementationVersion string             `json:"implementation_version"`
	LanguageInfo          KernelLanguageInfo `json:"language_info"`
	Banner                string             `json:"banner"`
	HelpLinks             []HelpLink         `json:"help_links"`
}

KernelInfo holds information about the kernel being served, for kernel_info_reply messages.

type KernelLanguageInfo

type KernelLanguageInfo struct {
	Name              string `json:"name"`
	Version           string `json:"version"`
	MIMEType          string `json:"mimetype"`
	FileExtension     string `json:"file_extension"`
	PygmentsLexer     string `json:"pygments_lexer"`
	CodeMirrorMode    string `json:"codemirror_mode"`
	NBConvertExporter string `json:"nbconvert_exporter"`
}

KernelLanguageInfo holds information about the language that this kernel executes code in.

type MIMEMap

type MIMEMap = map[string]any

MIMEMap holds data that can be presented in multiple formats. The keys are MIME types and the values are the data formatted with respect to its MIME type. All maps should contain at least a "text/plain" representation with a string value.

func EnsureMIMEMap

func EnsureMIMEMap(bundle MIMEMap) MIMEMap

type Message

type Message interface {
	// Error returns the error receiving the message, or nil if no error.
	Error() error

	// Ok returns whether there were no errors receiving the message.
	Ok() bool

	// ComposedMsg that started the current Message -- it's contained by Message.
	ComposedMsg() ComposedMsg

	// Kernel returns reference to the Kernel connections from where this Message was created.
	Kernel() *Kernel

	// Publish creates a new ComposedMsg and sends it back to the return identities over the
	// IOPub channel.
	Publish(msgType string, content interface{}) error

	// PromptInput sends a request for input from the front-end. The text in prompt is shown
	// to the user, and password indicates whether the input is a password (input shouldn't
	// be echoed in terminal).
	//
	// onInputFn is the callback function. It receives the original shell execute
	// message (m) and the message with the incoming input value.
	PromptInput(prompt string, password bool, onInput OnInputFn) error

	// CancelInput will cancel any `input_request` message sent by PromptInput.
	CancelInput() error

	// DeliverInput should be called if a message is received in Stdin channel. It will
	// check if there is any running process listening to it, in which case it is forwarded
	// (usually to the caller of PromptInput).
	// Still the dispatcher has to handle its delivery by calling this function.
	DeliverInput() error

	// Reply creates a new ComposedMsg and sends it back to the return identities over the
	// Shell channel.
	Reply(msgType string, content interface{}) error
}

Message is the interface of a received message. It includes an identifier that allows publishing back results to the identifier.

type MessageImpl

type MessageImpl struct {
	Composed   ComposedMsg
	Identities [][]byte
	// contains filtered or unexported fields
}

MessageImpl represents a received message or an Error, with its return identities, and a reference to the kernel for communication.

func (*MessageImpl) CancelInput

func (m *MessageImpl) CancelInput() error

CancelInput will cancel any `input_request` message sent by PromptInput.

func (*MessageImpl) ComposedMsg

func (m *MessageImpl) ComposedMsg() ComposedMsg

ComposedMsg that started the current Message -- it's contained by Message.

func (*MessageImpl) DeliverInput

func (m *MessageImpl) DeliverInput() error

DeliverInput should be called if a message is received in Stdin channel. It will check if there is any running process listening to it, in which case it is delivered. Still the user has to handle its delivery.

func (*MessageImpl) Error

func (m *MessageImpl) Error() error

Error returns the error receiving the message, or nil if no error.

func (*MessageImpl) Kernel

func (m *MessageImpl) Kernel() *Kernel

Kernel returns reference to the Kernel connections from where this Message was created.

func (*MessageImpl) Ok

func (m *MessageImpl) Ok() bool

Ok returns whether there were no errors receiving the message.

func (*MessageImpl) PromptInput

func (m *MessageImpl) PromptInput(prompt string, password bool, onInput OnInputFn) error

PromptInput sends a request for input from the front-end. The text in prompt is shown to the user, and password indicates whether the input is a password (input shouldn't be echoed in terminal).

onInputFn is the callback function. It receives the original shell execute message (m) and the message with the incoming input value.

func (*MessageImpl) Publish

func (m *MessageImpl) Publish(msgType string, content interface{}) error

Publish creates a new ComposedMsg and sends it back to the return identities over the IOPub channel.

func (*MessageImpl) Reply

func (m *MessageImpl) Reply(msgType string, content interface{}) error

Reply creates a new ComposedMsg and sends it back to the return identities over the Shell channel.

type MsgOrError

type MsgOrError struct {
	Msg zmq4.Msg
	Err error
}

MsgOrError describes an incoming message or a communication error. If an error occur consider closing the Kernel with Kernel.Stop().

type OnInputFn

type OnInputFn func(original, input *MessageImpl) error

OnInputFn is the callback function. It receives the original shell execute message and the message with the incoming input value.

type SocketGroup

type SocketGroup struct {
	ShellSocket   SyncSocket
	ControlSocket SyncSocket
	StdinSocket   SyncSocket
	IOPubSocket   SyncSocket
	HBSocket      SyncSocket
	Key           []byte
}

SocketGroup holds the sockets needed to communicate with the kernel, and the key for message signing.

type SubscriptionId added in v0.10.0

type SubscriptionId *list.Element

SubscriptionId is returned by Kernel.SubscribeInterrupt, and can be used by Kernel.UnsubscribeInterrupt.

type SyncSocket

type SyncSocket struct {
	Socket zmq4.Socket
	Lock   sync.Mutex
}

SyncSocket wraps a zmq socket with a lock which should be used to control write access.

func (*SyncSocket) RunLocked

func (s *SyncSocket) RunLocked(fn func(socket zmq4.Socket) error) error

RunLocked locks socket and runs `fn`.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL