Anthropic/AnthropicInterviewer
Viewer
•
Updated
•
1.25k
•
926
•
353
Adam Weilacher
Cockboy1
AI & ML interests
None yet
Recent Activity
new activity
about 2 months ago
InstantX/Qwen-Image-ControlNet-Inpainting:About Training Dataset
liked
a dataset
about 2 months ago
Anthropic/AnthropicInterviewer
upvoted
a
changelog
about 2 months ago
HuggingChat for Docs
Organizations
None yet
About Training Dataset
🔥
1
2
#7 opened about 2 months ago
by
Queen-Historia
liked
a
dataset
about 2 months ago
upvoted
a
changelog
about 2 months ago
Changelog
HuggingChat for Docs
•
114
I asked about Monica's Dress, and a cigar.
reacted to
martinsu's
post with 🔥👀
about 2 months ago
Post
3312
https://huggingface.co/blog/martinsu/potus-broke-my-pipeline
How POTUS Completely Broke My Flash 2.5-Based Guardrail
Did quite a bit of deep research on this one, since it IMHO matters. At first I used this story to amuse fellow MLOps guys, but then I went deeper and was surprised.
To those who don't want to read too much, in plain English: when you give the model a high-stakes statement that clashes with what it "knows" about the world, it gets more brittle. Sometimes to a point of being unusable.
Or an even shorter version: do not clash with the model's given worldview—it will degrade to some extent.
And in practice, it means that in lower-resource languages like Latvian and Finnish (and probably others), Flash 2.5 is an unreliable guardrail model when something clashes with the model's general "worldview".
However, I'm sure this degradation applies to other languages and models as well to varying extents.
In one totally normal week of MLOps, my news summarization pipeline started failing intermittently. Nothing was changed. No deploys. No prompt edits. No model version bump (as far as I could tell). Yet the guardrail would suddenly turn into a grumpy judge and reject outputs for reasons that felt random, sometimes even contradicting itself between runs. It was the worst kind of failure: silent, flaky, and impossible to reproduce on demand.
Then I noticed the pattern: it started when one specific named entity appeared in the text — Donald Trump ** (**and later in tests — Bernie Sanders too ).
And then down the rabbit hole I went.
How POTUS Completely Broke My Flash 2.5-Based Guardrail
Did quite a bit of deep research on this one, since it IMHO matters. At first I used this story to amuse fellow MLOps guys, but then I went deeper and was surprised.
To those who don't want to read too much, in plain English: when you give the model a high-stakes statement that clashes with what it "knows" about the world, it gets more brittle. Sometimes to a point of being unusable.
Or an even shorter version: do not clash with the model's given worldview—it will degrade to some extent.
And in practice, it means that in lower-resource languages like Latvian and Finnish (and probably others), Flash 2.5 is an unreliable guardrail model when something clashes with the model's general "worldview".
However, I'm sure this degradation applies to other languages and models as well to varying extents.
In one totally normal week of MLOps, my news summarization pipeline started failing intermittently. Nothing was changed. No deploys. No prompt edits. No model version bump (as far as I could tell). Yet the guardrail would suddenly turn into a grumpy judge and reject outputs for reasons that felt random, sometimes even contradicting itself between runs. It was the worst kind of failure: silent, flaky, and impossible to reproduce on demand.
Then I noticed the pattern: it started when one specific named entity appeared in the text — Donald Trump ** (**and later in tests — Bernie Sanders too ).
And then down the rabbit hole I went.
-
- 3 replies
liked
a
dataset
4 months ago
liked
a
model
4 months ago
liked
a
Space
4 months ago
published
a
Space
4 months ago
pip install polars
#114 opened 4 months ago
by
Cockboy1
Rename markdown/apache-2.0.md to markdown/apache-2.0.md/CC-BY-NC-ND
1
#74 opened 4 months ago
by
Cockboy1
liked
a
dataset
4 months ago
Update README.md
1
#5 opened 4 months ago
by
Cockboy1
Update README.md
❤️
🚀
1
2
#4 opened 4 months ago
by
Cockboy1
liked
a
Space
4 months ago
published
a
Space
4 months ago
updated
a
dataset
4 months ago
liked
a
dataset
4 months ago
published
a
dataset
4 months ago