mewld
A generic discord bot clusterer. Will be open source to help other large bot developers.
Mewld clusterer is rather generic and any bot should be able to adapt to it without any changes to mewld
What is a cluster
A cluster is a way to scale large discord bots by breaking them into seperate processes, each with a set of shards. mewld
is an implementation of this system.
Terminology
- Cluster - A single process with a set of shards in it
- Cluster size - The amount of shards in a cluster
- Embedding - Embedding
mewld
into a Go project for increased customizability
How it works
-
Shard count is retrieved using Get Gateway Bot
. This is then used to create a ClusterMap
assigning each cluster a set of shards based on per_cluster
in config.yaml. These ClusterMap
's are then used to make a InstanceList
of Instance
structs.
-
First instance is then started, once this instance sends launch_next
, the next instance is then launched until every cluster is launched, the last cluster should also send a launch_next
once fully ready
-
mewld
also handles other tasks such as cluster management via its webui which also comes with (upcoming) support for application command permission configs (for private commands that should only be visible to specific roles in a support or staff server)
-
mewld
supports max_concurrency
under the following limits. Firstly, the underlying bot must support launching several shards concurrently (as mewld
does not actually start shards). Secondly, the number of shards that can actually be
Redis
Mewld uses redis for communication with clusters and for action logs. Action logs are stored as a Redis list under ${redis_channel_name}/actlogs
.
Data Format:
JSON with the following structure (golang syntax to make updating docs simpler)
Scope string `json:"scope"`
Action string `json:"action"`
Args map[string]any `json:"args,omitempty"`
CommandId string `json:"command_id,omitempty"`
Output any `json:"output,omitempty"`
Data map[string]any `json:"data,omitempty"` // Used in action logs
Operations
Syntax |
Description |
Args |
launch_next |
Ready to launch next cluster, if available |
id -> cluster ID |
rollingrestart |
Rolling restart all clusters, one by one |
|
start |
Starts a cluster with a specific ID |
id -> cluster ID |
stop |
Starts a cluster with a specific ID |
id -> cluster ID |
statuses |
Gets the statuses of all clusters. |
|
shutdown |
Shuts down the entire bot and lets systemctl start it up again if needed |
|
restartproc |
Shuts down bot with error code so systemctl restarts it automatically |
|
diag |
The cluster must respond with a proc.DiagResponse payload. This is used as a diagnostic by the clusterer and may be used in the future for more important actions. |
|
Diagnostics
Whenever a diagnostic is sent over the $CHANNEL
channel, the cluster must respond with a diag
(see Operations table) within 10 seconds. A diagnostic can be identified based on the existence of a diag
key
FAQ
- Why are there so many
PingCheckStop <- i.ClusterID
?
Answer: To ensure that no erroneous ping checks are still running in background thus leading to random cluster death.