Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
Paper • 2607.07508 • Published • 31
Seydoux
Articles liés au Seydoux - Articles related to Seydoux
-
-
Proximal Policy Optimization Algorithms
Paper • 1707.06347 • Published • 12 -
Scaling Laws for Neural Language Models
Paper • 2001.08361 • Published • 12 -
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
Paper • 2012.13255 • Published • 6 -
Compacter: Efficient Low-Rank Hypercomplex Adapter Layers
Paper • 2106.04647 • Published • 2 -
LoRA: Low-Rank Adaptation of Large Language Models
Paper • 2106.09685 • Published • 65 -
Evaluating Large Language Models Trained on Code
Paper • 2107.03374 • Published • 12 -
Training language models to follow instructions with human feedback
Paper • 2203.02155 • Published • 26 -
Training Compute-Optimal Large Language Models
Paper • 2203.15556 • Published • 13 -
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Paper • 2205.14135 • Published • 16 -
Emergent Abilities of Large Language Models
Paper • 2206.07682 • Published • 4 -
FP8 Formats for Deep Learning
Paper • 2209.05433 • Published • 1 -
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Paper • 2210.09261 • Published • 2 -
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
Paper • 2210.17323 • Published • 12 -
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Paper • 2303.10512 • Published • 5 -
QLoRA: Efficient Finetuning of Quantized LLMs
Paper • 2305.14314 • Published • 64 -
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Paper • 2305.18290 • Published • 71 -
Let's Verify Step by Step
Paper • 2305.20050 • Published • 12 -
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Paper • 2306.00978 • Published • 15 -
Knowledge Distillation of Large Language Models
Paper • 2306.08543 • Published • 24 -
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
Paper • 2306.13649 • Published • 38 -
Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
Paper • 2307.05695 • Published • 25 -
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
Paper • 2307.08691 • Published • 10 -
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning
Paper • 2308.03303 • Published • 4 -
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
Paper • 2309.12307 • Published • 90 -
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Paper • 2310.06770 • Published • 13 -
VeRA: Vector-based Random Matrix Adaptation
Paper • 2310.11454 • Published • 31 -
A General Theoretical Paradigm to Understand Learning from Human Preferences
Paper • 2310.12036 • Published • 20 -
FP8-LM: Training FP8 Large Language Models
Paper • 2310.18313 • Published • 34 -
Tied-Lora: Enhacing parameter efficiency of LoRA with weight tying
Paper • 2311.09578 • Published • 17 -
GPQA: A Graduate-Level Google-Proof Q&A Benchmark
Paper • 2311.12022 • Published • 39 -
A Rank Stabilization Scaling Factor for Fine-Tuning with LoRA
Paper • 2312.03732 • Published • 13 -
LoRAMoE: Revolutionizing Mixture of Experts for Maintaining World Knowledge in Language Model Alignment
Paper • 2312.09979 • Published • 3 -
KTO: Model Alignment as Prospect Theoretic Optimization
Paper • 2402.01306 • Published • 24 -
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Paper • 2402.03300 • Published • 149 -
DistiLLM: Towards Streamlined Distillation for Large Language Models
Paper • 2402.03898 • Published • 5 -
DoRA: Weight-Decomposed Low-Rank Adaptation
Paper • 2402.09353 • Published • 33 -
LoRA+: Efficient Low Rank Adaptation of Large Models
Paper • 2402.12354 • Published • 8 -
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Paper • 2403.03507 • Published • 192 -
ORPO: Monolithic Preference Optimization without Reference Model
Paper • 2403.07691 • Published • 74 -
PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models
Paper • 2404.02948 • Published • 5 -
SimPO: Simple Preference Optimization with a Reference-Free Reward
Paper • 2405.14734 • Published • 13 -
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
Paper • 2405.17604 • Published • 4 -
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
Paper • 2406.01574 • Published • 57 -
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Paper • 2407.08608 • Published • 2 -
On the Impact of Fine-Tuning on Chain-of-Thought Reasoning
Paper • 2411.15382 • Published • 1 -
Kimi k1.5: Scaling Reinforcement Learning with LLMs
Paper • 2501.12599 • Published • 132 -
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
Paper • 2501.12948 • Published • 463 -
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Paper • 2501.17161 • Published • 127 -
s1: Simple test-time scaling
Paper • 2501.19393 • Published • 128 -
LIMO: Less is More for Reasoning
Paper • 2502.03387 • Published • 63 -
How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?
Paper • 2502.14502 • Published • 93 -
DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
Paper • 2503.07067 • Published • 34 -
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Paper • 2503.14476 • Published • 148 -
SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
Paper • 2503.18892 • Published • 32 -
Understanding R1-Zero-Like Training: A Critical Perspective
Paper • 2503.20783 • Published • 61 -
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
Paper • 2504.05118 • Published • 27 -
Qwen3 Technical Report
Paper • 2505.09388 • Published • 344 -
HD-PiSSA: High-Rank Distributed Orthogonal Adaptation
Paper • 2505.18777 • Published • 2 -
Reinforcement Fine-Tuning Naturally Mitigates Forgetting in Continual Post-Training
Paper • 2507.05386 • Published • 2 -
Group Sequence Policy Optimization
Paper • 2507.18071 • Published • 323 -
InfiR2: A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models
Paper • 2509.22536 • Published • 3 -
PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
Paper • 2511.21285 • Published • 2 -
TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies
Paper • 2511.23225 • Published • 3 -
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Paper • 2601.11868 • Published • 38 -
Jet-RL: Enabling On-Policy FP8 Reinforcement Learning with Unified Training and Rollout Precision Flow
Paper • 2601.14243 • Published • 25 -
LongCat-Flash-Thinking-2601 Technical Report
Paper • 2601.16725 • Published • 184 -
FP8-RL: A Practical and Stable Low-Precision Stack for LLM Reinforcement Learning
Paper • 2601.18150 • Published • 11 -
GLM-5: from Vibe Coding to Agentic Engineering
Paper • 2602.15763 • Published • 216 -
FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling
Paper • 2603.05451 • Published • 2 -
Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels
Paper • 2603.22276 • Published • 15 -
DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Paper • 2606.19348 • Published • 40 -
Fara-1.5: Scalable Learning Environments for Computer Use Agents
Paper • 2606.20785 • Published • 9 -
AsyncOPD: How Stale Can On-Policy Distillation Be?
Paper • 2606.24143 • Published • 32 -
MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training
Paper • 2606.30406 • Published • 25 -
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
Paper • 2607.08393 • Published • 19 -
Kimi K3: Open Frontier Intelligence
Paper • 2607.24653 • Published • 508 -
Reinforcement Learning for Code Optimization
Paper • 2607.25970 • Published • 13 -
Antares: Foundation Models for Agentic Vulnerability Localization
Paper • 2608.02407 • Published • 3 -
On-Policy Self-Distillation without Any Supervision
Paper • 2608.06296 • Published • 218 -
Apodex 1.1: Scaling Agentic Intelligence for Complex Work
Paper • 2608.23283 • Published • 197 -
WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report
Paper • 2608.24053 • Published • 62 -
Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence
Paper • 2608.21156 • Published • 54 -
AutoSaddler: Automatic Harness Optimization with Durable Updates from Agent Execution Traces
Paper • 2608.23041 • Published • 58 -
On-Policy Self-Distillation in Diffusion Models
Paper • 2608.24646 • Published • 55