README
¶
Duplicate Detector (Golang)
Introduction
Memcached-based duplicate detector.
Given a message id/hash, it can check whether it was previously seen by the system.
Installation
# load dependency
$ go get github.com/bradfitz/gomemcache/memcache
# install package
$ go get github.com/quipo/duplicatedetector
Sample usage
package main
import (
"fmt"
"github.com/bradfitz/gomemcache/memcache"
"github.com/quipo/duplicatedetector"
)
func main() {
mc := memcache.New("10.0.0.1:11211", "10.0.0.2:11211")
prefix := "myapp."
dupedetector := duplicatedetector.NewChecker(mc, prefix, 3600) // cache for 1 hour
for i := 0; i < 3; i++ {
dupe, err := dupedetector.IsDuplicate("abc")
if nil != err {
fmt.Println("**** ERROR from memcache: ", err)
}
if dupe {
fmt.Println("Entry is a duplicate: skipping")
continue
}
fmt.Println("First time this entry is seen")
}
}
Author
Lorenzo Alberton
- Web: http://alberton.info
- Twitter: @lorenzoalberton
- Linkedin: /in/lorenzoalberton
Copyright
See LICENSE document
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Checker ¶
type Checker struct {
// contains filtered or unexported fields
}
checker wraps some utility methods to work with memcached as duplicate detector
func NewChecker ¶
NewChecker returns a new instance of a Duplicate Detector It can be configured with a special prefix (useful for namespacing different apps) and a TTL (in seconds, either a relative time from now, up to 1 month, or an absolute Unix epoch time. Zero means the items have no expiration time)
func (*Checker) Delete ¶
Delete will remove the item from the cache, allowing a new Item with the same key in
func (*Checker) Has ¶
Has will check if the item has been previously seen already The function could return an error in case Memcache is not reachable or the retrieved value is not what was stored by the duplicate detector
func (*Checker) IsDuplicate ¶
IsDuplicate checks if the ID has been seen before (true) or if it's the first time (false). This counts as a touch: the first time an ID is checked, it is added to the cache; the second time the same ID is checked, it is considered as a duplicate The function could return an error in case Memcache is not reachable
Source Files