drmaa2interface

package module
v1.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 1, 2024 License: BSD-2-Clause Imports: 3 Imported by: 101

README

drmaa2interface

DRMAA2 compatible native Go interfaces and structs for building DRMAA2 compatible middleware

Why using drmaa2interface?

This repository simplifies the process to create Go DRMAA2 wrappers for job schedulers, process managers, resource management systems etc. (like for starting up OS processes, workflows, containers, pods...).

What is DRMAA2?

DRMAA2 is an acronym for Distributed Resource Management Application API version 2 which is an open and freely usable standard defined by the Open Grid Forum.

Unlike other standards it is a common subset of functions available in all major DRMs (like Univa Grid Engine, LSF, SLURM, Condor, PBS).

More information

More information can be found at the DRMAA website.

Please feel free to create issues on github.

Projects

The DRMAA2 interface is used in following projects:

Documentation

Overview

Package drmaa2interface implements the DRMAA2 Go interface. Actual Go DRMAA2 compatible implementations should use that interface to guarantee compatibility.

Index

Constants

View Source 
const InfiniteTime = time.Duration(1<<63 - 1)

InfiniteTime is a special timeout value: Wait probably infinitly

View Source 
const PlaceholderHomeDir = "$DRMAA2_HOME_DIR$"

PlaceholderHomeDir is a specified string which can be used to request the home directory of the user (e.g. as the output directory of the job).

View Source 
const PlaceholderIndex = "$DRMAA2_INDEX$"

PlaceholderIndex is a specified string which can be used to put the output of a job in a file which has the array job task ID within the directory structure or filename.

View Source 
const PlaceholderWorkingDir = "$DRMAA2_WORKING_DIR$"

PlaceholderWorkingDir is a specified string which can be used to point to the current working directory of the job.

View Source 
const UnsetEnum = -1

UnsetEnum defines an unset enum (required?)

View Source 
const UnsetNum = -1

UnsetNum defines an unset number (which is different than 0)

View Source 
const UnsetTime = -3

UnsetTime is a special time value: Time or date not set

View Source 
const ZeroTime = time.Duration(0)

ZeroTime is a special timeout value: Don't wait

Variables

View Source 
var UnsetDict map[string]string

UnsetDict is means no dictionary is set (nil)

View Source 
var UnsetList []interface{}

UnsetList is used to differentiate between an empty and an unspecified (unset) list. Probably not needed in Go.

Functions

This section is empty.

Types

type ArrayJob 

type ArrayJob interface {
	GetID() string
	GetJobs() []Job
	GetSessionName() string
	GetJobTemplate() JobTemplate
	Suspend() error
	Resume() error
	Hold() error
	Release() error
	Terminate() error
}

ArrayJob defines all methods which can be executed on a DRMAA2 job array. A job array contains a set of jobs with the same properties (same job template).

type CPU 

type CPU int

CPU architecture types 

const (
	OtherCPU CPU = iota
	Alpha
	ARM
	ARM64
	Cell
	PARISC
	PARISC64
	X86
	X64
	IA64
	MIPS
	MIPS64
	PowerPC
	PowerPC64
	SPARC
	SPARC64
)

func CPUFromString added in v1.2.0 

func CPUFromString(cpu string) (CPU, bool)

func (CPU) String 

func (i CPU) String() string

type Capability 

type Capability int

Capability is a type which represents the availability of optional functionality of the DRMAA2 implementation. Optional functionality is defined by the DRMAA2 standard but not mandatory to implement. 

const (
	AdvanceReservation Capability = iota
	ReserveSlots
	Callback
	BulkJobsMaxParallel
	JtEmail
	JtStaging
	JtDeadline
	JtMaxSlots
	JtAccountingID
	RtStartNow
	RtDuration
	RtMachineOS
	RtMachineArch
)

func CapabilityFromString added in v1.2.0 

func CapabilityFromString(cap string) (Capability, bool)

func (Capability) String 

func (i Capability) String() string

type Error 

type Error struct {
	// Messages is a dynamically created human readable error description.
	// There is no fixed message catalog defined.
	Message string
	// ID is used for identifying the error type
	ID ErrorID
}

Error is a DRMAA2 error type (implements Go Error interface). All errors returned by any DRMAA2 method is (and can be casted to) this Error type.

func (Error) Error 

func (ce Error) Error() string

Error implements the Errorer interface.

func (Error) String 

func (ce Error) String() string

String implements the Stringer interface for an drmaa2.Error

type ErrorID 

type ErrorID int

ErrorID type represents a DRMAA2 standardized error ID which is part of the DRMAA2 Error type. 

const (
	Success ErrorID = iota
	DeniedByDrms
	DrmCommunication
	TryLater
	SessionManagement
	Timeout
	Internal
	InvalidArgument
	InvalidSession
	InvalidState
	OutOfResource
	UnsupportedAttribute
	UnsupportedOperation
	ImplementationSpecific
	LastError
)

func ErrorIDFromString added in v1.2.0 

func ErrorIDFromString(id string) (ErrorID, bool)

func (ErrorID) String 

func (i ErrorID) String() string

type Event 

type Event int

Event specifies the type of the event 

const (
	// NewState is set when a job state changed
	NewState Event = iota
	// Migrated is set when a job is re-located (to another host for example)
	Migrated
	// AttributeChange means that some job info member is changed
	AttributeChange
)

type EventChannel 

type EventChannel <-chan Notification

EventChannel sends Notifications about job events. When subscribed they need to be consumed in the rate created by the system.

type Extensible 

type Extensible interface {
	// ListExtensions lists all implementation specific key names for
	// a particular DRMAA2 extensible data type
	ListExtensions() []string
	DescribeExtension(string) string
}

Extensible is an interface which defines functions used to interact with extensible data structures (JobTemplate, JobInfo etc.).

type Extension 

type Extension struct {
	ExtensionList map[string]string `json:"extensionList,omitempty"` // stores the extension requests as string
}

Extension is a struct which is embedded in DRMAA2 objects which are extensible. The extension is named in the DRMAA2 spec as SetIntanceValue / GetInstanceValue.

type Job 

type Job interface {
	GetID() string
	GetSessionName() string
	GetJobTemplate() (JobTemplate, error)
	GetState() JobState
	GetJobInfo() (JobInfo, error)
	Suspend() error
	Resume() error
	Hold() error
	Release() error
	Terminate() error
	WaitStarted(time.Duration) error
	WaitTerminated(time.Duration) error
	Reap() error
}

Job defines all methods which needs to be implemented for a DRMAA job type.

type JobInfo 

type JobInfo struct {
	Extensible
	Extension         `json:"extension,omitempty"`
	ID                string        `json:"id,omitempty"`
	ExitStatus        int           `json:"exitStatus,omitempty"`
	TerminatingSignal string        `json:"terminationSignal,omitempty"`
	Annotation        string        `json:"annotation,omitempty"`
	State             JobState      `json:"state,omitempty"`
	SubState          string        `json:"subState,omitempty"`
	AllocatedMachines []string      `json:"allocatedMachines,omitempty"`
	SubmissionMachine string        `json:"submissionMachine,omitempty"`
	JobOwner          string        `json:"jobOwner,omitempty"`
	Slots             int64         `json:"slots,omitempty"`
	QueueName         string        `json:"queueName,omitempty"`
	WallclockTime     time.Duration `json:"wallclockTime,omitempty"`
	CPUTime           int64         `json:"cpuTime,omitempty"`
	SubmissionTime    time.Time     `json:"submissionTime,omitempty"`
	DispatchTime      time.Time     `json:"dispatchTime,omitempty"`
	FinishTime        time.Time     `json:"finishTime,omitempty"`
}

JobInfo represents the state of a job. 

var UnsetJobInfo *JobInfo

UnsetJobInfo means no job info is set (nil)

func CreateJobInfo 

func CreateJobInfo() (ji JobInfo)

CreateJobInfo creates a JobInfo object where all values are initialized with UNSET (needed in order to differentiate if a value is not set or 0).

type JobSession 

type JobSession interface {
	Close() error
	GetContact() (string, error)
	GetSessionName() (string, error)
	GetJobCategories() ([]string, error)
	GetJobs(ji JobInfo) ([]Job, error)
	GetJobArray(id string) (ArrayJob, error)
	RunJob(jt JobTemplate) (Job, error)
	RunBulkJobs(jt JobTemplate, begin int, end int, step int, maxParallel int) (ArrayJob, error)
	WaitAnyStarted(jobs []Job, timeout time.Duration) (Job, error)
	WaitAnyTerminated(jobs []Job, timeout time.Duration) (Job, error)
}

JobSession contains all functions for managing jobs within one names job session. Multiple concurrent job sessions can be used at one point in time. A job session is a logical concept for separating workflows.

type JobState 

type JobState int

JobState represents the state of a job. 

const (
	Unset JobState = iota
	Undetermined
	Queued
	QueuedHeld
	Running
	Suspended
	Requeued
	RequeuedHeld
	Done
	Failed
)

func JobStateFromString added in v1.2.0 

func JobStateFromString(state string) (JobState, bool)

func (JobState) String 

func (i JobState) String() string

type JobTemplate 

type JobTemplate struct {
	Extensible        `xml:"-" json:"-"`
	Extension         `json:"extension,omitempty"`
	RemoteCommand     string            `json:"remoteCommand,omitempty"`
	Args              []string          `json:"args,omitempty"`
	SubmitAsHold      bool              `json:"submitAsHold,omitempty"`
	ReRunnable        bool              `json:"reRunnable,omitempty"`
	JobEnvironment    map[string]string `json:"jobEnvironment,omitempty"`
	WorkingDirectory  string            `json:"workingDirectory,omitempty"`
	JobCategory       string            `json:"jobCategory,omitempty"`
	Email             []string          `json:"email,omitempty"`
	EmailOnStarted    bool              `json:"emailOnStarted,omitempty"`
	EmailOnTerminated bool              `json:"emailOnTerminated,omitempty"`
	JobName           string            `json:"jobName,omitempty"`
	InputPath         string            `json:"inputPath,omitempty"`
	OutputPath        string            `json:"outputPath,omitempty"`
	ErrorPath         string            `json:"errorPath,omitempty"`
	JoinFiles         bool              `json:"joinFiles,omitempty"`
	ReservationID     string            `json:"reservationID,omitempty"`
	QueueName         string            `json:"queueName,omitempty"`
	MinSlots          int64             `json:"minSlots,omitempty"`
	MaxSlots          int64             `json:"maxSlots,omitempty"`
	Priority          int64             `json:"priority,omitempty"`
	CandidateMachines []string          `json:"candidateMachines,omitempty"`
	MinPhysMemory     int64             `json:"minPhysMemory,omitempty"`
	MachineOs         string            `json:"machineOs,omitempty"`
	MachineArch       string            `json:"machineArch,omitempty"`
	StartTime         time.Time         `json:"startTime,omitempty"`
	DeadlineTime      time.Time         `json:"deadlineTime,omitempty"`
	StageInFiles      map[string]string `json:"stageInFiles,omitempty"`
	StageOutFiles     map[string]string `json:"stageOutFiles,omitempty"`
	ResourceLimits    map[string]string `json:"resourceLimits,omitempty"`
	AccountingID      string            `json:"accountingString,omitempty"`
}

JobTemplate defines all fields for a job request defined by the DRMAA2 standard.

func (*JobTemplate) MarshalJSON added in v1.1.0 

func (jt *JobTemplate) MarshalJSON() ([]byte, error)

func (*JobTemplate) UnmarshalJSON added in v1.1.0 

func (jt *JobTemplate) UnmarshalJSON(data []byte) error

type Machine 

type Machine struct {
	Extensible
	Extension      `json:"extension,omitempty"`
	Name           string  `json:"name,omitempty"`
	Available      bool    `json:"available,omitempty"`
	Sockets        int64   `json:"sockets,omitempty"`
	CoresPerSocket int64   `json:"coresPerSocket,omitempty"`
	ThreadsPerCore int64   `json:"threadsPerCore,omitempty"`
	Load           float64 `json:"load,omitempty"`
	PhysicalMemory int64   `json:"physicalMemory,omitempty"`
	VirtualMemory  int64   `json:"virtualMemory,omitempty"`
	Architecture   CPU     `json:"architecture,omitempty"`
	OSVersion      Version `json:"osVersion,omitempty"`
	OS             OS      `json:"os,omitempty"`
}

Machine represents a compute instance implementing the extension interface.

type MonitoringSession 

type MonitoringSession interface {
	CloseMonitoringSession() error
	GetAllJobs(filter JobInfo) ([]Job, error)
	GetAllQueues(names []string) ([]Queue, error)
	GetAllMachines(names []string) ([]Machine, error)
	GetAllReservations() ([]Reservation, error)
}

MonitoringSession interface defines all methods required for implementing a DRMAA2 compatible monitoring session.

type Notification 

type Notification struct {
	Evt         Event    `json:"event"`
	JobID       string   `json:"jobID"`
	SessionName string   `json:"sessionName"`
	State       JobState `json:"jobState"`
}

Notification is the argument of the callback function automatically called for an event.

type OS 

type OS int

OS is the operating system type. 

const (
	OtherOS OS = iota
	AIX
	BSD
	Linux
	HPUX
	IRIX
	MacOS
	SunOS
	TRU64
	UnixWare
	Win
	WinNT
)

func OSFromString added in v1.2.0 

func OSFromString(os string) (OS, bool)

func (OS) String 

func (i OS) String() string

type Queue 

type Queue struct {
	Extensible
	Extension `json:"extension,omitempty"`
	Name      string `json:"name,omitempty"`
}

Queue implements all required elements of a queue.

type Reservation 

type Reservation interface {
	GetID() (string, error)
	GetSessionName() (string, error)
	GetTemplate() (ReservationTemplate, error)
	GetInfo() (ReservationInfo, error)
	Terminate() error
}

Reservation implements all methods required to be a valid DRMAA2 compatible reservation, created by a ReservationSession.

type ReservationInfo 

type ReservationInfo struct {
	Extensible
	Extension            `json:"extension,omitempty"`
	ReservationID        string    `json:"reservationID,omitempty"`
	ReservationName      string    `json:"reservationName,omitempty"`
	ReservationStartTime time.Time `json:"reservationStartTime,omitempty"`
	ReservationEndTime   time.Time `json:"reservationEndTime,omitempty"`
	ACL                  []string  `json:"acl,omitempty"`
	ReservedSlots        int64     `json:"reservedSlots,omitempty"`
	ReservedMachines     []string  `json:"reservedMachines,omitempty"`
}

ReservationInfo contains all details of the current state of a resource reservation.

type ReservationSession 

type ReservationSession interface {
	Close() error
	GetContact() (string, error)
	GetSessionName() (string, error)
	GetReservation(string) (Reservation, error)
	RequestReservation(ReservationTemplate) (Reservation, error)
	GetReservations() ([]Reservation, error)
}

ReservationSession provides all methods required for a DRMAA2 compatible reservation session.

type ReservationTemplate 

type ReservationTemplate struct {
	Extensible
	Extension         `json:"extension,omitempty"`
	Name              string        `json:"name,omitempty"`
	StartTime         time.Time     `json:"startTime,omitempty"`
	EndTime           time.Time     `json:"endTime,omitempty"`
	Duration          time.Duration `json:"duration,omitempty"`
	MinSlots          int64         `json:"minSlots,omitempty"`
	MaxSlots          int64         `json:"maxSlots,omitempty"`
	JobCategory       string        `json:"jobCategory,omitempty"`
	UsersACL          []string      `json:"userACL,omitempty"`
	CandidateMachines []string      `json:"candidateMachines,omitempty"`
	MinPhysMemory     int64         `json:"minPhysMemory,omitempty"`
	MachineOs         string        `json:"machineOs,omitempty"`
	MachineArch       string        `json:"machineArch,omitempty"`
}

ReservationTemplate contains ressource requests for a resource reservation.

type SessionManager 

type SessionManager interface {
	CreateJobSession(name, contact string) (JobSession, error)
	CreateReservationSession(sessionName, contact string) (ReservationSession, error)
	OpenMonitoringSession(sessionName string) (MonitoringSession, error)
	OpenJobSession(sessionName string) (JobSession, error)
	OpenReservationSession(name string) (ReservationSession, error)
	DestroyJobSession(sessionName string) error
	DestroyReservationSession(sessionName string) error
	GetJobSessionNames() ([]string, error)
	GetReservationSessionNames() ([]string, error)
	GetDrmsName() (string, error)
	GetDrmsVersion() (Version, error)
	Supports(Capability) bool
	RegisterEventNotification() (EventChannel, error)
}

SessionManager defines all methods available from a DRMAA2 compatible SessionManager implementation. A SessionManager handles creation, opening, closing, destruction of DRMAA2 sessions. Sessions can be job sessions, monitoring sessions, or optionally reservation sessions. It provides generic methods for querying supported optional functionality and versioning.

type Version 

type Version struct {
	Major string `json:"major,omitempty"`
	Minor string `json:"minor,omitempty"`
}

Version is a DRMAA2 version type which holds the major version and minor version.

func (Version) String 

func (v Version) String() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.