runner

runner

Introduction

runner is the runner of pipego written in Go.

Prerequisites

• Go >= 1.18.0

Run

version=latest make build
./bin/runner --listen-url=:29090

Docker

version=latest make docker
docker run ghcr.io/pipego/runner:latest --listen-url=:29090

Usage

usage: runner --listen-url=LISTEN-URL [<flags>]

pipego runner


Flags:
  --[no-]help              Show context-sensitive help (also try --help-long and --help-man).
  --[no-]version           Show application version.
  --listen-url=LISTEN-URL  Listen URL (host:port)
  --log-level="INFO"       Log level (DEBUG|INFO|WARN|ERROR)

Protobuf

1. Task

{
  "apiVersion": "v1",
  "kind": "runner",
  "metadata": {
    "name": "runner"
  },
  "spec": {
    "task": {
      "name": "task",
      "file": {
        "content": "bytes",
        "gzip": true
      },
      "params": [
        {
          "name": "env",
          "value": "val"
        }
      ],
      "commands": [
        "cmd",
        "argv"
      ],
      "log": {
        "width": 500
      }
    }
  }
}

task.file: script file in bash

task.file.content: bytes

The shebang in task.file.content should be one of below:

#!/bin/bash

#!/usr/bin/env bash

task.file.gzip: boolean

task.params: parameter and value

name1=value1 ($name1: value1)

name2=$name1 ($name2: value1)

name3=$name2 ($name3: value1)

name4=$$name1 ($name4: 790name1, the pid of this script)

name4=${BASHPID}name1 ($name4: 790name1, the pid of current instance)

name5=#name1 ($name5: #name1, invalid symbol in Bash)

task.commands: command and argument

task.log: task log

task.log.width: line width in runes (default: 500)

Output

{
  "pos": 1,
  "time": "1136214245000000000",
  "message": "text"
}

pos: line position

time: unix timestamp

message: line message in string

The tag in the line and file as below:

BOL: break of line

EOF: end of file

2. Glance

{
  "apiVersion": "v1",
  "kind": "runner",
  "metadata": {
    "name": "runner"
  },
  "spec": {
    "glance": {
      "dir": {
        "path": "/path/to/name"
      },
      "file": {
        "path": "/path/to/name",
        "maxSize": 1000
      },
      "sys": {
        "enable": true
      }
    }
  }
}

glance.dir: list directory contents

glance.file: fetch file content in base64

glance.file.maxSize: maximum file size in bytes

glance.sys: show system info

glance.sys.enable: boolean

Output

{
  "dir": {
    "entries": [
      {
        "name": "name",
        "isDir": true,
        "size": 1000,
        "time": "2006-01-02 15:04:05",
        "user": "name",
        "group": "name",
        "mode": "drwxr-xr-x"
      }
    ]
  },
  "file": {
    "content": "base64",
    "readable": true
  },
  "sys": {
    "resource": {
      "allocatable": {
        "milliCPU": 16000,
        "memory": 12871671808,
        "storage": 269490393088
      },
      "requested": {
        "milliCPU": 12,
        "memory": 618688512,
        "storage": 19994185728
      }
    },
    "stats": {
      "cpu": {
        "total": "16 CPU",
        "used": "0%"
      },
      "host": "172.23.179.208",
      "memory": {
        "total": "11 GB",
        "used": "0 GB"
      },
      "os": "Ubuntu 20.04",
      "storage": {
        "total": "250 GB",
        "used": "18 GB"
      },
      "processes": [
        {
          "process": {
            "name": "init",
            "cmdline": "/init",
            "memory": 684032,
            "time": 1.00,
            "pid": 1
          },
          "threads": [
            {
              "name": "child",
              "cmdline": "/child",
              "memory": 684032,
              "time": 1.00,
              "pid": 2
            }
          ]
        }
      ]
    }
  },
  "error": "text"
}

3. Maint

{
  "apiVersion": "v1",
  "kind": "runner",
  "metadata": {
    "name": "runner"
  },
  "spec": {
    "maint": {
      "clock": {
        "sync": true,
        "time": 1257894000
      }
    }
  }
}

maint.clock: clock maintenance

maint.clock.sync: enable/disable clock synchronization

The clock synchronization on Ubuntu

sudo apt install -y ntp ntpdate ntpstat
sudo ntpdate -s time.nist.gov
sudo service ntp restart
ntpstat

maint.clock.time: clock base time (unix time)

Output

{
  "clock": {
    "sync": {
      "status": "synchronized"
    },
    "diff": {
      "time": 100,
      "dangerous": true
    }
  }
}

clock.sync: clock synchronization

clock.sync.status: clock synchronization status

synchronized: clock is synchronized

unsynchronized: clock is not synchronized

indeterminant: clock state is indeterminant

clock.diff: clock difference

clock.diff.time: clock difference in milliseconds

positive value: behind of time

negative value: ahead of time

clock.diff.dangerous: if the difference is big enough to be considered dangerous

License

Project License can be found here.

Reference

• argo-workflows
• asynq
• asynqmon
• bufio-example
• chanx
• cuelang
• cyclone-workflow
• dagger
• drone-dag
• drone-livelog
• drone-pipeline
• drone-runner
• gleam-workflow
• go-exec
• grpctest
• grpc-streaming
• jenkins-clockdifference
• jenkins-clockdifference
• kube-parallelize
• kube-schduler
• kube-scheduling
• kube-scheduling
• kube-scheduling
• kube-workqueue
• machinery
• termui
• websocket-command
• wiki-dag