twitch_chat_filter

package module
v0.0.0-...-22c433a Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2017 License: Apache-2.0 Imports: 11 Imported by: 0

README

IRC Client Filterer / Aggregator (for Twitch)

Twitch chat streams can become unreadable when there are many people typing at the same time. It is not uncommon to have Twitch streams with over 100,000 concurrent viewers -- when there are this many people in the same chat room, it can become quite difficult to parse anything meaningful from the chat stream.

I built this toy CLI UI app to filter out duplicate messages, and to monitor realtime statistics for twitch chat channels that have non trivial amounts of chat activity. Removing duplicates gets rid of common chat messages like emoticons and reactions (messages like "LUL", "Kappa", etc), leaving behind more relevant messages.

You can see the app in action by clicking the image below: Twitch Chat Filterer

Functionality

Explanation of widgets in this app:

  1. Chat History - history of chat messages.
  2. Chat Box - lets you type and send messages.
  3. Message Rate Graph - shows the rate of messages in the channel for a given time window (few minutes and past hour)
  4. Message Stats - shows min, max, and avg messages for a given time window (few minutes and past hour)
  5. Trending Messages - shows duplicate messages, sorted by number of occurrences for a given time window (10s and 60s)

Usage:

This app requires golang to be installed. I could precompile binaries, but that seems excessive since this is not much more than a toy app.

  • Get a twitch account, login, and go to http://twitchapps.com/tmi/ to generate a password token.

  • Clone this repo, and cd into the cmd folder, and run

go run main.go -c=<twitch_channel_name> -s=irc.twitch.tv -w=<password_token> -u=<username>

making sure to replace twitch_channel_name, password_token, and username.

Help Text

This is shown when running the application without passing the required flags.

Aggregates common messages within a given time window

Usage:
  twitch_chat_filter [flags]

Flags:
  -c, --channel string                  Required. Irc Channel to join
  -d, --filter-duplicate-ttl duration   If true, will not show messages that have already been chatted recently. (default 10s)
  -f, --filter-duplicates               If true, will not show messages that have already been chatted recently. (default true)
  -h, --help                            help for twitch_chat_filter
  -s, --irc-server-host string          Required. Endpoint of irc server to connect to (default "irc.chat.twitch.tv")
  -p, --irc-server-port string          Port of irc-server-host (default "6667")
  -w, --password string                 IRC password to connect with
  -t, --tls-port string                 Port for TLS connections (default "443")
  -l, --ttl duration                    Sliding time window in seconds of how long to keep counts of messages in the trending bucket (default 10s)
  -u, --user string                     Required. IRC username to connect with

Future?

There may be ways of improving the filtering in the future, using some form of streaming TF-IDF for example, (see Streaming Trend Detection in Twitter), or doing something simpler with n-gram and stop-word checking, but until I actually have a reason to implement those things, this app will be as is for now.

Documentation

Index

Constants

View Source 
const (
	Increment = iota
	Decrement = iota
)

updateAction Types

Variables

View Source 
var (
	IrcServerHost, IrcServerPort, TlsPort, IrcUser, IrcPassword, IrcChannel string
	MessageTTL, FilterDuplicateTTL                                          time.Duration
	FilterDuplicates                                                        bool
)
View Source 
var TwitchCmd = &cobra.Command{
	Use:   "twitch_chat_filter",
	Short: "Filters twitch chat stream",
	Long:  `Aggregates common messages within a given time window`,
}

Functions

func Max 

func Max(x, y int) int

func Min 

func Min(x, y int) int

func RegisterIrcAndUiHandlers 

func RegisterIrcAndUiHandlers(c *irc.Conn) (quits chan bool, msgs chan *IrcMessage, typings chan string)

func ReplaceText 

func ReplaceText(input string) string

func Start 

func Start(cmd *cobra.Command, args []string)

Types

type ChatAggregator 

type ChatAggregator struct {
	TTL            time.Duration
	Widget         *ui.List
	SortedMessages *SortedMessages
}

func NewChatAggregator 

func NewChatAggregator(textColor ui.Attribute, ttl time.Duration) *ChatAggregator

type ChatBox 

type ChatBox struct {
	Widget         *ui.Par
	DefaultMessage string
	Data           string
	Input          <-chan string
}

func NewChatBox 

func NewChatBox(input <-chan string, messages chan<- *IrcMessage, c *irc.Conn) *ChatBox

type ChatHistory 

type ChatHistory struct {
	Widget              *ui.List
	ChatHistoryData     []string
	Size                int
	FilteredMessagesSet *ccache.Cache
}

func NewChatHistory 

func NewChatHistory() *ChatHistory

func (*ChatHistory) UpdateAndRender 

func (self *ChatHistory) UpdateAndRender(msg *IrcMessage)

type IrcMessage 

type IrcMessage struct {
	Nick    string
	Message string
}

type MessageCount 

type MessageCount struct {
	Message string
	Count   int
}

type MessageRateSparkline 

type MessageRateSparkline struct {
	Widget  *ui.Sparkline
	Counter *SlidingWindowCounter
}

func NewMessageRateSparkline 

func NewMessageRateSparkline(color ui.Attribute, counter *SlidingWindowCounter) *MessageRateSparkline

func (*MessageRateSparkline) IncrementAndRender 

func (self *MessageRateSparkline) IncrementAndRender(sparklines *ui.Sparklines)

type MessageStatsChart 

type MessageStatsChart struct {
	Widget  *ui.BarChart
	Counter *SlidingWindowCounter
}

func NewMessageStatsChart 

func NewMessageStatsChart(barColor, numColor ui.Attribute, counter *SlidingWindowCounter) *MessageStatsChart

func (*MessageStatsChart) Render 

func (self *MessageStatsChart) Render()

type SlidingWindowCounter 

type SlidingWindowCounter struct {
	Data       []int
	Min        int
	Max        int
	Avg        int
	Sum        int
	Stats      []int // contains Min, Max, Avg, Sum
	Window     time.Duration
	NumWindows int
}

func NewSlidingWindowCounter 

func NewSlidingWindowCounter(timeWindow time.Duration, numWindows int) (*SlidingWindowCounter, error)

func (*SlidingWindowCounter) Increment 

func (self *SlidingWindowCounter) Increment()

type SortedMessages 

type SortedMessages struct {
	Data               []MessageCount // no size limit
	View               []string       // size limit applies
	ViewSize           int
	ViewCountThreshold int

	NotifyViewChange chan bool // an element is put in here every time the view changes
	// contains filtered or unexported fields
}

func NewSortedMessages 

func NewSortedMessages(size, viewThreshold int) *SortedMessages

func (*SortedMessages) Decrement 

func (self *SortedMessages) Decrement(message string, delay time.Duration)

func (*SortedMessages) Increment 

func (self *SortedMessages) Increment(message string, ttl time.Duration)

type UILayoutState 

type UILayoutState struct {
	Header                *ui.Par
	ChatHistory           *ChatHistory
	ChatBox               *ChatBox
	MinuteStats           *MessageStatsChart
	HourStats             *MessageStatsChart
	MessageRateWidget     *ui.Sparklines
	MinuteMessageRate     *MessageRateSparkline
	HourMessageRate       *MessageRateSparkline
	SecondsChatAggregator *ChatAggregator
	MinuteChatAggregator  *ChatAggregator
	MessageStream         <-chan *IrcMessage
	IrcConn               *irc.Conn
}

func NewUILayoutState 

func NewUILayoutState() *UILayoutState

func (*UILayoutState) InitBodyAndLoop 

func (self *UILayoutState) InitBodyAndLoop()

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.