👋 Open to Work

RDTvlokip PRO

RDTvlokip

1 98 3

https://rdtvlokip.fr

AI & ML interests

None yet

Recent Activity

reacted to theirpost with 🤗 about 4 hours ago

I tried to teach a network to write using nothing but reward. No pretraining, no data, random weights. It worked twice. Both wins were fake. Test 1, copy a fixed sentence: solved in 1,639 episodes. Then I checked why. The per-position reward splits 12^12 into twelve independent 12-armed bandits. It doesn't guide the search, it deletes it. Transfer to a target sharing no positions was slower than starting over. Test 2, a grammar judged by a hand-written parser: 99.9% grammatical. Then I forced the antecedent and measured the consequent. P(noun agrees | determiner) = 0.333. Exactly 2 determiners out of 6. It learned no rule at all. It found an all-plural sublanguage where agreement is vacuously true. Then the one that stung: an untrained network plus a filter beats 20,000 episodes of REINFORCE on both validity and diversity. Nobody runs that baseline because it looks too stupid to bother with. None of this is new. Reward shaping moving the optimum: Ng, Harada & Russell, 1999. The variance decomposition that retires the word "sparse": Fisher's ANOVA. The real lesson: A high score on a verifier you wrote yourself measures your specification, not the agent. It finds the corner of the output space where your constraint is vacuous, and it looks like success from every angle except the one you forgot to check. Code, 8 figures, and the notebook with every refuted hypothesis 👇 🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-network-to-write-with-reward-only 💻 https://github.com/RDTvlokip/RDTRL

repliedto their post about 4 hours ago

reacted to salma-remyx's post with 🔥 about 13 hours ago

VQASynth adds a code-as-action annotation agent Data synthesis frameworks may distill a pipeline of expert models through templates or LLM re-phrasing, but templates limit the variety of the training data. Wanting more flexibility for VQASynth, we've added an experimental SpatialAnnotator agent with scene understanding tools-as-methods, built on NVIDIA-labs OO Agents (NOOA): https://github.com/NVIDIA-NeMo/labs-OO-Agents In NOOA an agent is a Python class. Fields hold state, docstrings are prompts, type annotations are contracts, and a method body of `...` becomes an LLM-driven loop. Scene understanding tools stay ordinary deterministic methods. ``` from nooa import Agent class SpatialAnnotator(Agent, llm=llm): """Answer spatial questions about an image using scene understanding tools.""" scene: SceneContext # masks, depth, intrinsics, canonical transform # Deterministic Python. Bodies omitted here. def measure(self, a, b=None, kind="object_to_object") -> Measurement def count(self, target: str) -> Count # Agentic. The runtime hands this to the model. async def annotate(self, image: Image, question: str) -> SpatialAnswer: """Answer the spatial question. This is offline labeling, so there is no human available for clarification. Prefer a canonical interpretation, and decline if the question cannot be grounded.""" ... ``` GitHub: https://github.com/remyxai/VQASynth#agent-with-vqasynth-tools

View all activity

Organizations

reacted to their post with 🤗 about 4 hours ago

Post

228

I tried to teach a network to write using nothing but reward. No pretraining, no data, random weights. It worked twice.

Both wins were fake.

Test 1, copy a fixed sentence: solved in 1,639 episodes. Then I checked why. The per-position reward splits 12^12 into twelve independent 12-armed bandits. It doesn't guide the search, it deletes it. Transfer to a target sharing no positions was slower than starting over.

Test 2, a grammar judged by a hand-written parser: 99.9% grammatical. Then I forced the antecedent and measured the consequent. P(noun agrees | determiner) = 0.333. Exactly 2 determiners out of 6. It learned no rule at all. It found an all-plural sublanguage where agreement is vacuously true.

Then the one that stung: an untrained network plus a filter beats 20,000 episodes of REINFORCE on both validity and diversity. Nobody runs that baseline because it looks too stupid to bother with.

None of this is new. Reward shaping moving the optimum: Ng, Harada & Russell, 1999. The variance decomposition that retires the word "sparse": Fisher's ANOVA.

The real lesson:

A high score on a verifier you wrote yourself measures your specification, not the agent. It finds the corner of the output space where your constraint is vacuous, and it looks like success from every angle except the one you forgot to check.

Code, 8 figures, and the notebook with every refuted hypothesis 👇

🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-network-to-write-with-reward-only
💻 https://github.com/RDTvlokip/RDTRL

2 replies

replied to their post about 4 hours ago

Your numbers are right and I reproduce all of them. Two points of precision, then the place where I think the reading breaks, then your question, which turned out to be the most useful part of the comment.

Exact values. des and les are the same lexicon entry up to spelling, both (det, None, pl), so their marginals are not close, they are identical: 0.30888… on the short grammar, 0.15327783… on the long one. Your 0.1536 / 0.1535 split is Monte Carlo noise. Singulars are 0.27555… and 0.13714… And your mechanism is right: E[accord_det_nom | det] is 3/4 for a plural determiner against 1/2 for a singular one, and the None is what buys it.

Now run the same probe on position 1. Short grammar, E[R | x_p = t], uniform elsewhere, closed form:

pos 0 (det)   des 0.3089  les 0.3089  |  le la un une 0.2756      +0.0333  PLURAL
pos 1 (noun)  chat chien fleur table 0.2944  |  chats ... 0.2778  +0.0167  SINGULAR
pos 2 (verb)  all six 0.2789                                       flat

The two positions point in opposite directions, and the two causes are orthogonal, one per feature:

                             genre   number   mean
  credit to a singular noun    2/3      2/3    2/3
  credit to a plural noun      2/3      1/3    1/2
  credit to 'le'  (det sg)     1/2      1/2    1/2
  credit to 'les' (det pl)      1       1/2    3/4

The plural determiner's edge is entirely in genre, so it is your None. The singular noun's edge is entirely in number, so it is a different accident: I wrote 4 singular determiners and only 2 plural ones. Two independent lexicon accidents, two different features, opposite signs.

That is why the order-1 greedy sequence is invalid:

'des chat chante'  ->  INVALID, R = 0.8333

If the coordinate-wise greedy optimum is not a solution, the coordinates are contradicting each other. That is the signature, and it is the sentence my article never wrote. It gives both tables and it gives the invalid greedy sequence, and it never says why the sequence is invalid. You read position 0, stopped, and drew exactly the inference my text authorised. That is a writing defect on my side, not a measurement one.

And position 1 is the one that names the branch. Determiner, adjective and verb all agree with the noun. The noun is the only feature carrier, so "all-plural sublanguage" is a claim about the noun's number, not the determiner's.

Which position wins? I ran it, and it is the noun.

Exact gradient, same GRU, same objective E[R] + β·H, but p(s) obtained by enumerating all 8000 sequences, so zero sampling noise:

  beta   seed    valid%   modes    sg%     pl%
  0.01   0,1,2   100.00    12.0   100.0    0.0
  0.02   0        99.95    24.0   100.0    0.0
  0.02   1        99.93    12.0   100.0    0.0
  0.02   2        99.94    24.0   100.0    0.0

Six of six. Take the noise away and it goes singular every time. Sampled REINFORCE at β = 0, pure reward, no entropy bonus, 3 seeds: 100% singular, 1.0 effective mode, all three. Full sweep, 8 β values by 3 seeds, 24 runs: 15 singular, 9 plural. At your β = 0.02 specifically, seed 0 is plural at 18.6 modes (the run you are quoting), seeds 1 and 2 are singular at 11.7 and 12.0.

So: the collapse is settled before episode 1, and I agree. The corner is not. Which vacuous corner it lands in is a seed lottery running about 2:1 toward singular. Your closed-form direction is real, it is just outvoted by the position it does not look at.

The diversity number cannot arbitrate either, because the singular sublanguage also holds exactly 24 of the 48 (four determiners at 6 each, against les 12 plus des 12). 18.6 is 77.5% of 24, but seeds 1 and 2 sit at ~49% of their 24 at the same β.

Your question about the long grammar. I had not run it, and I should have. Its low validity had me filing it as a scaling failure rather than as a collapse worth analysing, which was a mistake. Five seeds, β = 0.08, 40k samples each:

  seed  valid%   distinct valid   number of the noun, valid mass
    0     6.98        144              100.0% sg
    1     9.41        144              100.0% sg
    2     6.79        151               99.7% sg
    3     7.54        144              100.0% pl
    4     7.27        148               99.9% sg

It does collapse to a single number family. 144 distinct valid sentences is exactly one family out of 288, the split being perfectly symmetric. And the family is singular in 4 seeds of 5. Same answer as the short grammar, same direction, and the plural determiner edge loses again.

Structurally the vacuous corner is not even there. Adjectives spell out all four genre-number combinations with no None anywhere, so E[accord_adj_nom | noun] is exactly 0.5 for every noun and the adjective marginal is perfectly flat (0.1329 for all eight). Going plural now buys the free genre on the determiner alone, one subscore of four instead of one of three. Both edges shrink by exactly (3/4)(20/31) = 0.4839: det +0.0333 to +0.0161, noun +0.0167 to +0.0081. Adding an agreement rule added denominator and no counter-signal. The order-1 greedy sequence becomes des grand chat chante bien, still invalid.

Where you are simply right: the probe belongs before training, it costs one enumeration, and it is a protocol step rather than a post-hoc analysis. It already existed in the repo (gradient_exact.py, part 1, which prints both tables) but buried in an analysis script, which is why I did not reach for it when writing the article. It is now its own thing, src/test2_grammar/sonde_ordre1.py, all positions, both grammars, plus the long-grammar measurement above, credited to you in the docstring.

The lesson I would now write on it is not "find the vacuous corner". It is compute every position, because the corner is where they disagree, and the sign of the disagreement is not readable from the first token.

reacted to salma-remyx's post with 🔥 about 13 hours ago

Post

1520

from nooa import Agent

    class SpatialAnnotator(Agent, llm=llm):
        """Answer spatial questions about an image using scene understanding tools."""

        scene: SceneContext          # masks, depth, intrinsics, canonical transform

        # Deterministic Python. Bodies omitted here.
        def measure(self, a, b=None, kind="object_to_object") -> Measurement
        def count(self, target: str) -> Count

        # Agentic. The runtime hands this to the model.
        async def annotate(self, image: Image, question: str) -> SpatialAnswer:
            """Answer the spatial question. This is offline labeling, so there is
            no human available for clarification. Prefer a canonical
            interpretation, and decline if the question cannot be grounded."""
            ...

GitHub: https://github.com/remyxai/VQASynth#agent-with-vqasynth-tools

upvoted an article 1 day ago

Article

🎲 Apprendre à un réseau à écrire avec seulement une récompense — il a atteint 99,9 % grammatical sans apprendre une seule règle 🇫🇷

RDTvlokip

•

1 day ago

• 3

published an article 1 day ago

Article

🎲 Apprendre à un réseau à écrire avec seulement une récompense — il a atteint 99,9 % grammatical sans apprendre une seule règle 🇫🇷

RDTvlokip

•

1 day ago

• 3

posted an update 1 day ago

Post

228

I tried to teach a network to write using nothing but reward. No pretraining, no data, random weights. It worked twice.

Both wins were fake.

Test 1, copy a fixed sentence: solved in 1,639 episodes. Then I checked why. The per-position reward splits 12^12 into twelve independent 12-armed bandits. It doesn't guide the search, it deletes it. Transfer to a target sharing no positions was slower than starting over.

Test 2, a grammar judged by a hand-written parser: 99.9% grammatical. Then I forced the antecedent and measured the consequent. P(noun agrees | determiner) = 0.333. Exactly 2 determiners out of 6. It learned no rule at all. It found an all-plural sublanguage where agreement is vacuously true.

Then the one that stung: an untrained network plus a filter beats 20,000 episodes of REINFORCE on both validity and diversity. Nobody runs that baseline because it looks too stupid to bother with.

None of this is new. Reward shaping moving the optimum: Ng, Harada & Russell, 1999. The variance decomposition that retires the word "sparse": Fisher's ANOVA.

The real lesson:

A high score on a verifier you wrote yourself measures your specification, not the agent. It finds the corner of the output space where your constraint is vacuous, and it looks like success from every angle except the one you forgot to check.

Code, 8 figures, and the notebook with every refuted hypothesis 👇

🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-network-to-write-with-reward-only
💻 https://github.com/RDTvlokip/RDTRL

2 replies

upvoted an article 1 day ago

Article

🎲 Teaching a network to write with reward only — it hit 99.9% grammatical without learning a single rule 🇫🇷

RDTvlokip

•

1 day ago

• 1

published an article 1 day ago

Article

🎲 Teaching a network to write with reward only — it hit 99.9% grammatical without learning a single rule 🇫🇷

RDTvlokip

•

1 day ago

• 1

replied to PhysiQuanty's post 23 days ago

💪

upvoted 2 articles 25 days ago

Article

🧊 The machine froze: the hardware wall behind my multi-seed validation 🇫🇷

RDTvlokip

•

25 days ago

• 2

Article

🧊 La machine a gelé : le mur matériel derrière ma validation multi-seed 🇫🇷

RDTvlokip

•

25 days ago

• 2

published an article 25 days ago

Article

🧊 La machine a gelé : le mur matériel derrière ma validation multi-seed 🇫🇷

RDTvlokip

•

25 days ago

• 2

replied to their post 25 days ago

Straight answer to your question first: the absolute-threshold variant (Focal) is per-token with no global budget, depth is emergent, and on real text ~86% hit the R=4 ceiling anyway. That arm can't be cleanly matched, you're right.

But the percentile variant (Nomade) already is a fixed budget by construction. q=0.3 freezes the 30% least-uncertain active tokens every iteration, which gives a deterministic depth cascade (≈30 / 21 / 15 / 34% to full depth) independent of the data. q is the passes-per-sequence knob you're asking for.

So the clean ablation falls straight out of that, exactly as you're framing it:

Entropy arm: percentile q=0.3, freeze the lowest-entropy 30% each iteration.
Random arm: freeze a random 30% each iteration.

Same q → identical depth distribution → identical pass budget. The only thing that differs is which tokens spend it. Any remaining gap is the selection signal, cleanly isolated, no confound from freeze count.

And it's a tiny change: swap the entropy-quantile mask for a random mask at the same fraction, same seed, same harness. That's article 4's core experiment now, matched-budget, selection is the only free variable.

Thanks you turned a vague "entropy vs random" into a properly controlled one. (Pending the hardware fix, but the design is locked.)

posted an update 25 days ago

Post

147

My 15M French LLM is finished. Three models on Hugging Face, clean code, honest results.

I can't finish validating them, because my PC won't stay on.

The last honest step was simple: rerun the three looped models on two more seeds, so I can report variance instead of a single lucky run. 8 runs, ~50 hours overnight. Routine.

Except mid-training, the whole machine freezes solid. Screen black, RAM still lit, power button dead, I have to cut mains power at the wall. No blue screen, no crash dump. Just a dead PC that's still powered.

The event logs tell the story: repeated NVIDIA driver errors (nvlddmkm 153, 4101) building up for days, even at idle, at 1am, with nothing running, then Kernel-Power 41 with no bugcheck. A 2017 GTX 1080 Ti fighting a 2026 driver that's known to be rough on Pascal cards.

Out of 8 planned runs, the machine survived exactly two. So the results stay exactly what I called them: preliminary, one seed. A crash is not a data point.

The frustrating part isn't the bug. It's that the science is clean and the code is ready, the only thing missing is a machine that survives a night.

The plan: roll back the driver, rent cloud GPUs (Lambda Labs) to finish the seeds, and long-term, build a rig that can actually sustain solo research. If you've beaten Kernel-Power 41 on a Pascal card or want to support the work. I'm genuinely open to it.

Solo ML runs on whatever hardware you can reach. Right now mine is a 2017 card that won't stay awake.

Full write-up 👇
🔗 https://huggingface.co/blog/RDTvlokip/the-hardware-wall-kernel-power-41

published an article 25 days ago

Article

🧊 The machine froze: the hardware wall behind my multi-seed validation 🇫🇷

RDTvlokip

•

25 days ago

• 2

replied to their post 25 days ago

Good catch. The ablation is clean and I hadn't run it. Maybe I'll add it to article 4 stochastic depth vs entropy halting, same harness, same protocol.

replied to their post 25 days ago

Hey, thanks for the feedback, everything you raise is already addressed in the article:

**Confirmation bias / "8-prompt spectacular":**dedicated section "Didn't you first report much bigger gains?", the 8×80 results are explicitly disclosed and corrected to 50×200, both sets of numbers visible.
Seed band: bolded disclaimer in the intro, repeated in limitations and conclusion: 1 seed, variance not controlled.
Halting on the 50-prompt rerun: dedicated Q&A: "Your adaptive halting barely fires at generation time. So what did it actually do?"

Worth a read before commenting.

New activity in RDTvlokip/Cadence-15M-fr 25 days ago

merci

🔥 1

#1 opened 26 days ago by

PhysiQuanty

posted an update 28 days ago

Post

2002

I finally changed the architecture of my 15M French LLM. It worked. Then I almost fooled myself about how much and catching that was the real win.

After proving last time that architecture is a threshold, not a lever, I got stubborn: could I change how the model learns? Four honest attempts, Lion, a sharper AdamW β2, multi-token prediction, LayerScale. Four failures. The bottleneck wasn't the learning rule either.

So I changed the shape of the computation instead: loop the same transformer blocks 4×, deeper reasoning, zero added parameters. It beat the baseline on perplexity, the first thing in the whole project to move that number. Then I added my own twist: let each token decide how deep to think, halting on its own entropy.

My first evaluation was spectacular. Coherence up 65%. Hallucinated names down 62%.

It was noise.

Eight prompts, one seed. I re-ran on 50 prompts × 200 tokens and watched the gains shrink to "modest" and on out-of-domain prompts, recurrence actually made things worse. No universal winner. And none of it is new: it's Adaptive Computation Time (2016), the Universal Transformer (2018), and LoopViT (2026), recombined and measured honestly.

The real lesson:

A number from 8 prompts is a rumor. The eval harness that kills your own best result is worth more than the result it kills. Cite your lineage. Stay preliminary until multiple seeds say otherwise.

The three models are live. The write-up is honest about every caveat 👇

🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-15m-french-llm-to-think-deeper

6 replies

upvoted an article 28 days ago

Article

🔁 Apprendre à un LLM français de 15M à penser plus profond — et à savoir quand s'arrêter 🇫🇷

RDTvlokip

•

28 days ago

• 3

RDTvlokip PRO

AI & ML interests

Recent Activity

Organizations

RDTvlokip's activity

🎲 Apprendre à un réseau à écrire avec seulement une récompense — il a atteint 99,9 % grammatical sans apprendre une seule règle 🇫🇷

🎲 Apprendre à un réseau à écrire avec seulement une récompense — il a atteint 99,9 % grammatical sans apprendre une seule règle 🇫🇷

🎲 Teaching a network to write with reward only — it hit 99.9% grammatical without learning a single rule 🇫🇷

🎲 Teaching a network to write with reward only — it hit 99.9% grammatical without learning a single rule 🇫🇷

🧊 The machine froze: the hardware wall behind my multi-seed validation 🇫🇷

🧊 La machine a gelé : le mur matériel derrière ma validation multi-seed 🇫🇷

🧊 La machine a gelé : le mur matériel derrière ma validation multi-seed 🇫🇷

🧊 The machine froze: the hardware wall behind my multi-seed validation 🇫🇷

merci

🔁 Apprendre à un LLM français de 15M à penser plus profond — et à savoir quand s'arrêter 🇫🇷