README
¶
moderation
moderation is a profanity filter for Go.
Goals
- Easy to use
- Minimum possible allocations, processing time, and binary size
- Minimum false negatives (including text like
h3110_w0r!d) - Minimum false positives
- (Experimental) Provide a way to censor text
- (Future) Other analysis types than inappropriate, profane, offensive, sexual, mean, spam (violence, contact info, etc.)
- (Future) Basic support for languages other than English
Example
package main
import (
"fmt"
"github.com/finnbear/moderation"
)
func main() {
printResult("hello world")
printResult("$#1t")
printResult("a$$")
printResult("assassin")
}
func printResult(phrase string) {
description := "is appropriate"
if moderation.IsInappropriate(phrase) {
description = "is NOT appropriate"
}
fmt.Printf("\"%s\" %s.\n", phrase, description)
}
$ go run hello_world.go
"hello world" is appropriate.
"$#1t" is NOT appropriate.
"a$$" is NOT appropriate.
"assassin" is appropriate.
Comparison
Accuracy was evaluated based on the first 100,000 items from this dataset of moderated comments.
| Package | Time | Accuracy | Comment |
|---|---|---|---|
| finnbear/moderation | 1.63s | 92.21% | Current API version is not stable |
| TwinProduction/go-away | 2.16s | 82.18% | Many false positives from combined words like "push it" |
Acknowledgements
- Radix implementation based on https://gitlab.com/caibear/go-boggle/
- Some profanities and test cases are based on https://github.com/TwinProduction/go-away
Documentation
¶
Overview ¶
The package moderation implements a profanity filter.
Index ¶
Examples ¶
Constants ¶
Variables ¶
var CensorReplacment rune = '*'
Functions ¶
func Censor ¶
Censor returns a string with all but the first character of any inappropriate segment replaced with CensorReplacment
It is currently Experimental and not fully tested
func IsInappropriate ¶
IsInappropriate returns whether a phrase contains enough inappropriate words to meet or exceed InappropriateThreshold
Equivalent to
moderation.Scan(text).Is(moderation.Inappropriate)
Example ¶
fmt.Println(IsInappropriate("hello"), IsInappropriate("sh1t"))
Output: false true
Types ¶
type Type ¶
type Type uint32
Types and severities of inappropriateness
For compability, always reference them by name as their value may change from version to version.
Use a bitwise OR of multiple profanity classifications, and a bitwise AND to specify a severity level (default Mild). The definition of Inappropriate (mildly profane, mildly offensive, mildly sexual, or severely mean) serves as a good example.
Other operations on Type's are NOT supported.
Severities sould be interpreteted on an "at least" basis, e.g. Mild means Mild, Moderate, OR Severe.
func Scan ¶ added in v0.10.0
Scan returns a bitmask of all types detected within given text, which can be queried with the Is function
Example ¶
result := Scan("you're a dumbass")
fmt.Println(result.Is(Profane), result.Is(Offensive), result.Is(Sexual), result.Is(Mean), result.Is(Mean&Severe))
Output: true false false true false
func (Type) Is ¶ added in v0.11.0
Is returns whether the scan result includes a given Type or set of Type's
Example (Severity) ¶
fmt.Println(Scan("sh1t").Is(Profane), Scan("sh1t").Is(Profane&Severe))
Output: true false
Example (Types) ¶
fmt.Println(Scan("shit").Is(Profane), Scan("shit").Is(Sexual|Mean))
fmt.Println(Scan("HELLO THERE").Is(Spam), Scan("duuuuuuuuumb").Is(Spam), Scan("Normal text").Is(Spam))
Output: true false true true false
Source Files
Directories