pollix/stuntd-support-triage
Text Classification • Updated • 4
Denis PRO
pollix
P(doom) 1%
AI & ML interests
None yet
Recent Activity
posted an update about 10 hours ago
stuntd 0.1.4 is out, small one.
Last time someone asked what the novelty gate does at the 0.99 quantile instead of 0.95. I checked: at 0.95 every head sends about 5% of totally normal traffic to the big model on purpose, and with three fields that adds up. So 0.99 is the default now.
It also ships examples/support/check.py. One command trains the demo heads twice (all templates, and with one template per category left out) and prints the whole sweep, screenshot is its output with the junk-input part cut. Every generalization number in the README comes from that script now, so you can rerun it instead of trusting me.
What it shows on the support demo:
- normal tickets answered locally: 70.6% at 0.99 vs 67.1% at 0.95 (72.3% with no gate), still 97% right
- tickets from templates the heads never saw: 0.3% answered locally at 0.99. Without the gate the heads answer 69% of those and get only 42% right
- the weather question, asdf and {} are stopped at every setting
Also auto_retrain finally works for Jev captures, it only counted OpenAI and Anthropic before.
```
pip install -U stuntd
```
https://github.com/bladedevoff/stuntd/releases/tag/v0.1.4 updated a model about 10 hours ago
pollix/stuntd-support-triage updated a Space about 10 hours ago
pollix/stuntdOrganizations
