Documentation ¶
Overview ¶
Package smartcb provides a circuit breaker based on https://github.com/rubyist/circuitbreaker that automatically adjusts the tripping error threshold based on abnormal increase in error rate. All you need to tell it is the nominal QPS ("queries per second") for your task and it automatically sets the best values for adjusting the circuit breaker's responsiveness. If you want, you can adjust the circuit breaker's sensitivity as per your situation.
The circuit breaker starts off with a learning phase for understanding the error profile of the wrapped command and then adjusts its tripping threshold for error rate based on what it has learned.
The error threshold is calculated as an exponential weighted moving average which smoothens out jitter and can detect rapid changes in error rate, allowing for the circuit to trip fast in case of a rapid degradation of the wrapped command.
Example ¶
// Initialise policies and set max. tolerable failure rate // to 15% (= 0.15) policies := smartcb.NewPolicies() policies.MaxFail = 0.15 // Create a SmartTripper Generator for a 10k QPS task var st = smartcb.NewSmartTripper(100000, policies) // Create a Circuit Breaker from the SmartTripper Generator var scb = smartcb.NewSmartCircuitBreaker(st) // The task to be wrapped with the circuit breaker protectedTask := func(errRate float64) (err error) { if rand.Float64() < errRate { return errors.New("forced error") } return nil } breakerEvents := scb.Subscribe() // Let's run the example for 200 ms stop := time.After(time.Millisecond * 200) loop := true for loop { select { case <-stop: // Stop execution now loop = false case e := <-breakerEvents: // Something changed with the circuit breaker if e == circuit.BreakerTripped { log.Println("Circuit Breaker tripped.", scb.ErrorRate(), st.State(), st.LearnedRate()) return } default: // Execute the task using circuit.Breaker.Call() method _ = scb.Call(func() error { return protectedTask(0.02) }, time.Second) } }
Output:
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func NewSmartCircuitBreaker ¶
func NewSmartCircuitBreaker(t *SmartTripper) *circuit.Breaker
NewSmartCircuitBreaker creates a new circuit.Breaker based on the SmartTripper
It returns a circuit.Breaker from github.com/rubyist/circuitbreaker Please see its documentation to understand how to use the breaker
Types ¶
type Policies ¶
type Policies struct { // Absolute highest failure rate above which the breaker must open // Default is 0.05 (5%). MaxFail float64 // Number of "decision windows" used for learning LearningWindowX float64 // Number of "decision windows" after which learning is restarted. // // This setting must be greater than LearningWindowX otherwise the breaker // would be in a perpetual learning state ReLearningWindowX float64 // Smoothing factor for error rate learning. Higher numbers reduce jitter // but cause more lag EWMADecayFactor float64 // Number of trials in a decision window. SamplesPerWindow int64 }
Policies for configuring the circuit breaker's decision making.
MaxFail is the only parameter that might need adjustment. Do not tweak the other parameters unless you are a statistician. If you must, experiment with changing one parameter at a time. All parameters are required to be > 0
func NewPolicies ¶
func NewPolicies() Policies
NewPolicies returns Policies initialised to default values
type SmartTripper ¶
type SmartTripper struct {
// contains filtered or unexported fields
}
SmartTripper controls behaviour of the tripping function used by the circuit breaker
All circuit breakers obtained out of a SmartTripper instance share their learning state, but the circuit breaker state (error rates, event counts, etc.) is not shared
func NewSmartTripper ¶
func NewSmartTripper(QPS int, p Policies) *SmartTripper
NewSmartTripper creates a SmartTripper based on the nominal QPS for your task
"Nominal QPS" is the basis on which the SmartTripper configures its responsiveness settings. A suitable value for this parameter would be your median QPS. If your QPS varies a lot during operation, choosing this value closer to max QPS will make the circuit breaker reluctant to trip during low traffic periods and choosing a value closer to min QPS will make it slow to respond during high traffic periods.
NOTE: Provide QPS value applicable for one instance of the circuit breaker, not the overall QPS across multiple instances.
func (*SmartTripper) LearnedRate ¶
func (t *SmartTripper) LearnedRate() float64
LearnedRate returns the tripping error rate learned by the SmartTripper
func (*SmartTripper) State ¶
func (t *SmartTripper) State() State
State returns the Learning/Learned status of the Smart Tripper
State change only happens when an error is triggered Therefore timing alone can not be relied upon to detect state changes