general_purpose_scheduler

General Purpose Scheduler Microservice

📄 Contents

• Contents
  • Description
  • Deployment
  • How does the general purpose scheduler microservice work?
  • Directory and file description
  • Common questions

❕ Description

The Scheduler service's goal is to schedule newly created pods.

❕ Deployment of the scheduler service

Before deploying the scheduler.yaml file, please configure the environment variables to the correct values used by the queue microservice.

env:
- name: MQ_HOST
  value: "sched-rabbitmq-0.sched-rabbitmq.custom-scheduler.svc.cluster.local"
- name: MQ_PORT
  value: "5672"
- name: MQ_USER
  value: "guest"
- name: MQ_PASS
  value: "guest"
- name: RECEIVE_QUEUE
  value: "epsilon.distributed"
- name: RETRY_QUEUE
  value: "epsilon.backoff"

RECEIVE_QUEUE indicates the queue the scheduler is going to be listening to for new pods send by the coordinator service.
RETRY_QUEUE indicates the queue the scheduler is going to send failed pods to.

❕ The General Purpose Scheduler Lifecycle

1. FETCH Stage

1. Wait for new pods to be send by the coordinator
2. When a new pod is received, proceed with fetching the details of the received pod from the Kube API Server.
3. Once the details of the pod is fetched form the Kube API server. The pod details can be send to the PreFilter stage.

2. PRE FILTER Stage

1. Send the pod through a list of preconfigured PreFilter Plugins
2. Once the pod passes all the checks by the PreFilter Plugins, the pod will be sent to the Filter Stage

3. FILTER Stage

1. Send the pod through a list of preconfigured Filter Plugins
2. Once the pod passes all the checks by the Filter Plugins, the pod will be sent to the PreScore Stage

4. PRE SCORE Stage

1. Send the pod through a list of preconfigured PreScore Plugins
2. Once all the PreScore plugins intitnlizes the required variables for use on the next stage, the pod will be sent to the Score Stage

5. SCORE Stage

1. Send the pod through a list of preconfigured Score Plugins
2. Once all the Score plugins return the score value, the pod will be sent to the Score Stage

6. BIND Stage

1. During the stage the scheduler will commit the changes to the cluster and ends the scheduling lifecycle. Only during this stage the pod is considered to be deployed.

❕ Directory and File description

❕ Common questions

Changing the list of plugins used by the scheduler

Modification of plugin details can be achieved by editing the /framework/v1alpha1/framework.go file under the NewFramework() function

Change a scheduler plugin's weight

Same as changing the list of plugins used by the scheduler shown above

Change scheduling lifecycle

Modification of the scheduling lifecycle can be achieved by editing the /scheduler/scheduler.go file under the Schedule() function

Adding a new scheduler plugin

In order to add a new scheduler plugin, follow the steps shown below

  1. Create a new folder in /framework/plugins
  2. Write the plugin implementation and store the file in the new folder created in 1
  3. Open /framework/plugins/registry.go and import the new folder that was created in 2
  4. Initilize the newly added plugin by calling its New function inside the NewInTreeRegistry() function
  5. Open /framework/v1alpha1/framework.go and add the plugin into the correct plugin list. 
     Each list is for a different stage, make sure to add the new plugin to the correct list