jobpool

Overview ¶

Package jobpool implements a pool of go routines that are dedicated to processing jobs posted into the pool. The jobpool maintains two queues, a normal processing queue and a priority queue. Jobs placed in the priority queue will be processed ahead of pending jobs in the normal queue.

If priority is not required, using ArdanStudios/workpool is faster and more efficient.

Read the following blog post for more information:blogspot
http://www.goinggo.net/2013/05/thread-pooling-in-go-programming.html

New Parameters ¶

The following is a list of parameters for creating a JobPool:

numberOfRoutines: Sets the number of job routines that are allowed to process jobs concurrently
queueCapacity:    Sets the maximum number of pending job objects that can be in queue

JobPool Management ¶

Go routines are used to manage and process all the jobs. A single Queue routine provides the safe queuing of work. The Queue routine keeps track of the number of jobs in the queue and reports an error if the queue is full.

The numberOfRoutines parameter defines the number of job routines to create. These job routines will process work subbmitted to the queue. The job routines keep track of the number of active job routines for reporting.

The QueueJob method is used to queue a job into one of the two queues. This call will block until the Queue routine reports back success or failure that the job is in queue.

Example Use Of JobPool ¶

The following shows a simple test application

package main

import (
    "github.com/goinggo/jobpool"
    "fmt"
    "time"
)

type WorkProvider1 struct {
    Name string
}

func (jobPool *WorkProvider1) RunJob(jobRoutine int) {

    fmt.Printf("Perform Job : Provider 1 : Started: %s\n", jobPool.Name)
    time.Sleep(2 * time.Second)
    fmt.Printf("Perform Job : Provider 1 : DONE: %s\n", jobPool.Name)
}

type WorkProvider2 struct {
    Name string
}

func (jobPool *WorkProvider2) RunJob(jobRoutine int) {

    fmt.Printf("Perform Job : Provider 2 : Started: %s\n", jobPool.Name)
    time.Sleep(5 * time.Second)
    fmt.Printf("Perform Job : Provider 2 : DONE: %s\n", jobPool.Name)
}

func main() {

    jobPool := jobpool.New(2, 1000)

    jobPool.QueueJob("main", &WorkProvider1{"Normal Priority : 1"}, false)

    fmt.Printf("*******> QW: %d  AR: %d\n", jobPool.QueuedJobs(), jobPool.ActiveRoutines())
    time.Sleep(1 * time.Second)

    jobPool.QueueJob("main", &WorkProvider1{"Normal Priority : 2"}, false)
    jobPool.QueueJob("main", &WorkProvider1{"Normal Priority : 3"}, false)

    jobPool.QueueJob("main", &WorkProvider2{"High Priority : 4"}, true)
    fmt.Printf("*******> QW: %d  AR: %d\n", jobPool.QueuedJobs(), jobPool.ActiveRoutines())

    time.Sleep(15 * time.Second)

    jobPool.Shutdown("main")
}

Example Output ¶

The following shows some sample output

*******> QW: 1  AR: 0
Perform Job : Provider 1 : Started: Normal Priority : 1
Perform Job : Provider 1 : Started: Normal Priority : 2
*******> QW: 2  AR: 2
Perform Job : Provider 1 : DONE: Normal Priority : 1
Perform Job : Provider 2 : Started: High Priority : 4