Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
pietrolesci
/
tokenizers
like
0
Transformers
arxiv:
1910.09700
Model card
Files
Files and versions
xet
Community
Deploy
Use this model
main
tokenizers
42.1 MB
1 contributor
History:
5 commits
pietrolesci
Upload folder using huggingface_hub
f634b8a
verified
3 months ago
_raw_tokenizers
Upload folder using huggingface_hub
3 months ago
bpe_dclm-edu150k_128k
Upload folder using huggingface_hub
3 months ago
.gitattributes
1.61 kB
Upload folder using huggingface_hub
3 months ago
README.md
Safe
5.17 kB
Upload tokenizer
3 months ago
raw_tok_path.txt
Safe
60 Bytes
Upload folder using huggingface_hub
3 months ago
special_tokens_map.json
Safe
29 Bytes
Upload tokenizer
3 months ago
tokenizer.json
Safe
9.27 MB
Upload tokenizer
3 months ago
tokenizer_config.json
Safe
403 Bytes
Upload tokenizer
3 months ago
