I think STE is to help humans examine model output especially its design, hence you can provide the right feedback
Yi Cui
onekq
AI & ML interests
Benchmark, Code Generation Model
Recent Activity
repliedto their post 3 days ago
I added ASD-STE100 to global claude.md repliedto their post 3 days ago
I added ASD-STE100 to global claude.md posted an update 4 days ago
I added ASD-STE100 to global claude.mdOrganizations
replied to their post 3 days ago
replied to their post 3 days ago
I transformed a wordy repo:
Size
- Words: 115,559 → 124,979 (+8.2%)
- Sentences: 4,795 → 10,983 (+129%)
Sentence length
- Average words per sentence: 24.1 → 11.4 (−53%)
- 90th-percentile words per sentence: 46 → 20 (−57%)
- Longest sentence: 175 → 62 words (−65%)
- Sentences over 20 words: 53.1% → 8.0% (−85%)
- Sentences over 25 words: 41.6% → 2.5% (−94%
- Sentences over 40 words: 15.8% → 0.2% (−99%)
Paragraphs
- Paragraphs with more than 6 sentences: 1.2%
Readability
- Flesch reading ease: 54.1 → 72.5 (+18 points)
- Flesch-Kincaid grade: 11.7 → 6.0 (−5.7 grad
Vocabulary and grammar
- Unique words: 7,233 → 5,796 (−20%)
- Passive voice per 1k words: 8.5 → 5.7 (−33%
- Complex tenses per 1k words: 1.5 → 0.5 (−66%)
- "-ing" words per 1k words: 26.9 → 12.2 (−55
Punctuation
- Semicolons per 1k words: 9.8 → 0.9 (−91%)
- Em dashes per 1k words: 8.9 → 0.1 (−99%)
replied to their post 13 days ago
Can't fine-tune or prompt engineer this model so you have to accept its accuracy as is. Add other harnesses if the accuracy is not as good
replied to their post 15 days ago
Really no idea. Just wait for the open source ones to come out
posted an update 17 days ago
Post
3199
My takes on Jev
1. Very likely a small model. You can certainly pretrain, but I would grab an existing base model, say Qwen 3 class
2. The new RL method is a breakthrough, classification doesn't need to align with human preferences
3. The new output is an overstatement. It's just a new LM head. Of course autoregressive decoding can be used for classification: it takes just a few tokens to express the output. Think twice: are you sure classification doesn't need few-shot, CoT, or reasoning? All of these depend on auto-regressiveness
4. It carves out a market already existing, which is now served by oversized LLMs (hence overpaid), e.g. LLM as judge, labeling
5. Jevons effect will kick in, promoting more modeling efforts for small budget teams. It might even accelerate RSI
1. Very likely a small model. You can certainly pretrain, but I would grab an existing base model, say Qwen 3 class
2. The new RL method is a breakthrough, classification doesn't need to align with human preferences
3. The new output is an overstatement. It's just a new LM head. Of course autoregressive decoding can be used for classification: it takes just a few tokens to express the output. Think twice: are you sure classification doesn't need few-shot, CoT, or reasoning? All of these depend on auto-regressiveness
4. It carves out a market already existing, which is now served by oversized LLMs (hence overpaid), e.g. LLM as judge, labeling
5. Jevons effect will kick in, promoting more modeling efforts for small budget teams. It might even accelerate RSI
- 7 replies
posted an update 19 days ago
Post
79
Not your weights, not your products.
This doesn't feel true at the beginning, but suddenly becomes true after PMF
This doesn't feel true at the beginning, but suddenly becomes true after PMF
- 1 reply
replied to their post 24 days ago
posted an update 24 days ago
Post
4489
I turned off subagents in Claude Code. Am I a minority?
- 5 replies
replied to their post about 1 month ago
Really appreciate everyone's knowledge sharing and critique here.
Isn't this a gloomy picture that no frontier model can be run on the edge? Smaller models are for workflows and automations.
We are all used to conversing with a frontier model and let it run things on your device. A weaker model will degrade your current experience hence no adoption.
replied to their post about 1 month ago
Wow! thx for all the works here. I wish more people can see the methodology.
My Mac is only 16GB 😅 We need to find one of those people above who have big budget
https://www.ebay.com/itm/257712960417
posted an update about 1 month ago
Post
1003
My take on device-side inference: it's all about high bandwidth memory (thinking about it, this holds for the cloud too).
MacBooks enjoy incidental capacity of apple silicon, but per-device RAM is too low (16 to 24GB), only sufficient for a decent SLM. 512GB is the highest you can go (Kimi K2*). Counting MLX downloads of Kimi K2* on Huggingface, I estimate the user base to be <25K.
On the other hand, the newly debuted DGX station (Nvidia) has 748GB, which can fit in the latest Kimi, DS, and Qwen. Also the quantization options of CUDA is way better than MLX.
For high-end inferencing, I place my bet on workstations over Macs.
MacBooks enjoy incidental capacity of apple silicon, but per-device RAM is too low (16 to 24GB), only sufficient for a decent SLM. 512GB is the highest you can go (Kimi K2*). Counting MLX downloads of Kimi K2* on Huggingface, I estimate the user base to be <25K.
On the other hand, the newly debuted DGX station (Nvidia) has 748GB, which can fit in the latest Kimi, DS, and Qwen. Also the quantization options of CUDA is way better than MLX.
For high-end inferencing, I place my bet on workstations over Macs.
- 7 replies
posted an update about 1 month ago
Post
167
My biggest takeaway from Ben Thompson interview is the legacy of this round of "bubble".
For railroad, it was the track, and dot com, fiber. Compared to these two, GPUs depreciate too fast.
The answer is power plants.
For railroad, it was the track, and dot com, fiber. Compared to these two, GPUs depreciate too fast.
The answer is power plants.
posted an update about 1 month ago
Post
2015
Back of the envelope calculation: Ox Alpha has been out for a week with 130K users and 7T tokens burned.
Method 1: Assuming Ox Alpha is the GLM 5.3 class, it has roughly the same active param count as DeepSeek V4 Pro (~40B vs 49B), derive the cost by the floor price DS has ever published.
Method 2: Assuming the users are concentrated within an 8-hour working window each day, derive number of H800 nodes needed (~700 nodes at $2/GPU-hour).
In both methods, I assume 90/10 IO split and 60% cache hit. Both methods come to $2M.
I think the ROI is awesome: (1) publicity and (2) data harvesting.
Method 1: Assuming Ox Alpha is the GLM 5.3 class, it has roughly the same active param count as DeepSeek V4 Pro (~40B vs 49B), derive the cost by the floor price DS has ever published.
Method 2: Assuming the users are concentrated within an 8-hour working window each day, derive number of H800 nodes needed (~700 nodes at $2/GPU-hour).
In both methods, I assume 90/10 IO split and 60% cache hit. Both methods come to $2M.
I think the ROI is awesome: (1) publicity and (2) data harvesting.
- 1 reply
replied to their post about 1 month ago
Whoever this is, they give away so much compute for a model launch
replied to their post about 1 month ago
Most likely
replied to their post about 1 month ago
Lots of SLMs, like the 27b one from Qwen