quantRunner

package

v4.10.3 Latest Latest Go to latest Published: May 1, 2024 License: Apache-2.0 Imports: 17 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/pixlise/core

Links

Open Source Insights

Documentation ¶

Rendered for

Overview ¶

Exposes interfaces and structures required to run PIQUANT in the Kubernetes cluster along with functions to access quantification files, logs, results and summaries of quant jobs.

Example (EstimateNodeCount) ¶

// Based on experimental runs in: https://github.com/pixlise/core/-/issues/113

// Can only use the ones where we had allcoation of 7.5 set in kubernetes, because the others weren't maxing out cores

// 5x11, 4035 PMCs, 3 elements, 8 cores (7.5 allocation in kubernetes), 10 nodes. Runtime: 3:22
fmt.Println(EstimateNodeCount(4035*2, 3, 3*60+22, 8, 50))
// 5x11, 4035 PMCs, 3 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 1:52 (with params)
fmt.Println(EstimateNodeCount(4035*2, 3, 60+52, 8, 50))
// 5x11, 4035 PMCs, 4 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:11 (with params)
fmt.Println(EstimateNodeCount(4035*2, 4, 2*60+11, 8, 50))
// 5x11, 4035 PMCs, 5 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:26 (with params)
fmt.Println(EstimateNodeCount(4035*2, 5, 2*60+26, 8, 50))
// 5x11, 4035 PMCs, 6 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:47 (with params)
fmt.Println(EstimateNodeCount(4035*2, 6, 2*60+47, 8, 50))
// 5x11, 4035 PMCs, 7 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 2:55 (no params though)
fmt.Println(EstimateNodeCount(4035*2, 7, 2*60+55, 8, 50))
// 5x11, 4035 PMCs, 8 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:12 (with params)
fmt.Println(EstimateNodeCount(4035*2, 8, 3*60+12, 8, 50))
// 5x11, 4035 PMCs, 9 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:23 (with params)
fmt.Println(EstimateNodeCount(4035*2, 9, 3*60+23, 8, 50))
// 5x11, 4035 PMCs, 10 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:35 (with params)
fmt.Println(EstimateNodeCount(4035*2, 10, 3*60+35, 8, 50))
// 5x11, 4035 PMCs, 11 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 3:46 (with params)
fmt.Println(EstimateNodeCount(4035*2, 11, 3*60+46, 8, 50))

// 5x5, 1769 PMCs, 11 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 1:47 (with params)
fmt.Println(EstimateNodeCount(1769*2, 11, 60+47, 8, 50))

// 5x5, 1769 PMCs, 4 elements, 8 cores (7.5 allocation in kubernetes), 20 nodes. Runtime: 0:59 (with params)
fmt.Println(EstimateNodeCount(1769*2, 4, 59, 8, 50))

// Ensure the max cores have an effect
fmt.Println(EstimateNodeCount(1769*2, 4, 59, 8, 6))

// It's a bit unfortunate we ran all but 1 tests on the same number of cores, but
// the above data varies the spectra count, element count and expected runtime

// Below we'd expect 20 for all answers except the first one, but there's a slight (and
// allowable) drift because we're not exactly spot on with our estimate, and there's
// fixed overhead time we aren't even calculating properly

Example (FilesPerNode) ¶

fmt.Println(FilesPerNode(8088, 5))
fmt.Println(FilesPerNode(8068, 3))

Output:

1619
2690

Index ¶

func EstimateNodeCount(spectraCount int32, elementCount int32, desiredRunTimeSec int32, ...) int32
func FilesPerNode(spectraCount int32, nodeCount int32) int32
func MakePMCFileName(nodeNumber int) string
type PiquantParams
type QuantRunner
- func GetQuantRunner(name string) (QuantRunner, error)

Constants ¶

This section is empty.

Variables ¶

This section is empty.

Functions ¶

func EstimateNodeCount ¶

func EstimateNodeCount(spectraCount int32, elementCount int32, desiredRunTimeSec int32, coresPerNode int32, maxNodes int32) int32

func FilesPerNode ¶

func FilesPerNode(spectraCount int32, nodeCount int32) int32

func MakePMCFileName ¶

func MakePMCFileName(nodeNumber int) string

MakePMCFileName - generates a name for the PMC list file based on a given node-numer

NOTE: nodes are 0-based whereas pmc filenames are 1-based,
	  so the first node is 0 and will work on pmcs in node00001.pmcs

Types ¶

type PiquantParams ¶

type PiquantParams struct {
	RunTimeEnv        string   `json:"runtimeEnv"`
	JobID             string   `json:"jobId"`
	JobsPath          string   `json:"jobsPath"`
	DatasetPath       string   `json:"datasetPath"`
	DetectorConfig    string   `json:"detectorConfig"`
	Elements          []string `json:"elements"`
	Parameters        string   `json:"parameters"`
	DatasetsBucket    string   `json:"datasetsBucket"`
	ConfigBucket      string   `json:"configBucket"`
	PiquantJobsBucket string   `json:"jobBucket"`
	QuantName         string   `json:"name"`
	PMCListName       string   `json:"pmcListName"`
	Command           string   `json:"command"`
}

PiquantParams - Parameters for running piquant, as generated by PIXLISE API

type QuantRunner ¶

type QuantRunner interface {
	RunPiquant(piquantDockerImage string, params PiquantParams, pmcListNames []string, cfg config.APIConfig, requestorUserId string, log logger.ILogger) error
}

func GetQuantRunner ¶

func GetQuantRunner(name string) (QuantRunner, error)

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL