quarks-container-run
Background & Motivation
Before Kubernetes, the processes of the jobs in an instance group were
managed by monit
.
This allowed a human operator to suspend (kill) and later restart
these processes as a means of preventing them from interfering with
low-level operations like restoring a cluster using raw database
backups, and the like. Such suspensions were also not visible at kube
level as the pod and container kept running, except through live- and
readiness-probes.
The process control features added to the quarks-container-run
helper
application of the operator serve the same purpose.
Interface
The process control features of quarks-container-run
are accessible through
an unix domain datagram socket at location
/var/vcap/data/
JOB/
PROCESS_containerrun.sock
in the
container. Due to this placement the feature is not accessible from
outside a cluster. An operator (or script written by such) has to log
into the relevant container(s) to use the feature.
-
Suspending the monitored child processes is done by sending the
command STOP
to this socket.
-
Conversely, restarting the child processes is done by sending the
command START
to this socket.
-
Sending a START
command when the child processes are running has
no effect. Conversely the same is true for sending a STOP
command when the child processes are suspended already.
-
Any other command sent to the socket is ignored.
Any tool able to send datagram packet to a unix domain socket of that
type should work.
Examples using netcat
:
echo START | nc -w 1 --unixsock --udp /var/vcap/data/JOB/PROCESS_containerrun.sock
echo STOP | nc -w 1 --unixsock --udp /var/vcap/data/JOB/PROCESS_containerrun.sock
Note that all of these sockets are placed in the volumne shared by all
container of all jobs of the instance group. It is enough to ssh into
one of the containers to be able to send commands to all
sockets and thus jobs.
Example:
for sock in $(find /var/vcap/data/ -name '*_containerrun.sock')
do
echo STOP | nc -w 1 --unixsock --udp $sock
done
Generating mocks
Run go generate ./...
to generate the mocks for the container-run packages.