Chatstral

The problem

Server chat moderation with keyword lists only catches what you thought to list. Modern abuse is misspellings, coded language, and context — exactly the things a pattern list can’t see. The obvious fix is an LLM, but the usual answer is a cloud API, which means latency, cost, and sending your players’ chat to someone else’s servers.

Chatstral runs the LLM locally, on the machine already hosting the server.

How it works

• Local LLM: a lightweight 3B-parameter model (Shieldstral) served via llama.cpp’s llama-server, auto-downloaded per-platform.
• Instant blacklist: keyword/regex blocking runs first, so obvious spam is rejected with zero latency.
• AI scoring: logprob analysis — P(yes) vs P(no) for “is this harmful?” — rather than a brittle yes/no classification.

// Score is softmax over the logprobs of "yes" / "no" tokens.
double pYes = Math.exp(logprobYes) / (Math.exp(logprobYes) + Math.exp(logprobNo));
boolean block = pYes > config.aiThreshold;

• Async + batching: messages are queued and sent to llama-server in batches of up to 8 every 100ms, so a slow model never blocks chat.

Architecture

ChatFilter.java        - Bukkit event listener, cooldown management
ShieldstralClient.java - HTTP client for llama-server, score parsing, batching
ModelManager.java      - downloads + manages the llama-server lifecycle
Blacklist.java         - word/regex matching
Chatstral.java         - main plugin, command registration

Deployment realism

Cross-platform llama-server binaries are fetched and cached automatically, the model is streamed from HuggingFace with progress bars, and every setting (ai-threshold, cooldown-ms, port) is in a config.yml the server admin can edit live with /chatstral reload.

The honest constraint: the ~1.3GB model and ~2GB RAM requirement mean this is aimed at servers that already have a bit of headroom — the tradeoff for keeping moderation fully local and free.