TokSuite

toksuite's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Articles

Malikeh1375 

authored 3 papers 2 days ago

Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation

Paper • 2410.08371 • Published Oct 10, 2024 • 3

DEPAC: a Corpus for Depression and Anxiety Detection from Speech

Paper • 2306.12443 • Published Jun 20, 2023

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

Paper • 2512.20757 • Published 4 days ago • 14

craffel 

authored a paper 2 days ago

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

Paper • 2512.20757 • Published 4 days ago • 14

gsaltintas 

authored a paper 2 days ago

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

Paper • 2512.20757 • Published 4 days ago • 14

mciccone 

authored a paper 2 days ago

TokSuite: Measuring the Impact of Tokenizer Choice on Language Model Behavior

Paper • 2512.20757 • Published 4 days ago • 14

gsaltintas 

updated 8 datasets 3 days ago

toksuite/tokenizer_robustness_completion_stem

Viewer • Updated 3 days ago • 614 • 213

toksuite/tokenizer_robustness_completion_italian

Viewer • Updated 3 days ago • 1.09k • 406

toksuite/tokenizer_robustness_completion_english

Viewer • Updated 3 days ago • 1.14k • 557

toksuite/tokenizer_robustness_completion_math

Viewer • Updated 3 days ago • 189 • 351

toksuite/tokenizer_robustness_completion_chinese

Viewer • Updated 3 days ago • 485 • 651

toksuite/tokenizer_robustness_completion_turkish

Viewer • Updated 3 days ago • 621 • 269

toksuite/tokenizer_robustness_completion_farsi

Viewer • Updated 3 days ago • 747 • 146

toksuite/tokenizer_robustness_completion_general

Viewer • Updated 3 days ago • 68 • 81

gsaltintas 

updated 6 models 3 days ago

toksuite/supertoken_models-llama_google-gemma-2-2b

Text Generation • 2B • Updated 3 days ago • 239

toksuite/supertoken_models-llama_meta-llama-Llama-3.2-1B

Text Generation • 2B • Updated 3 days ago • 97

toksuite/supertoken_models-llama_CohereLabs-aya-expanse-8b

Text Generation • 2B • Updated 3 days ago • 54

toksuite/supertoken_models-llama_tiktoken-gpt-4o

Text Generation • 2B • Updated 3 days ago • 56

toksuite/supertoken_models-llama_common-pile-comma-v0.1

Text Generation • 2B • Updated 3 days ago • 73

toksuite/supertoken_models-llama_microsoft-Phi-3-mini-4k-instruct

Text Generation • 1B • Updated 3 days ago • 74