AxisCommunity/DuckyModel-bucket / tokenizer_config.json
AxisCommunity's picture
download
raw
195 Bytes
{
"backend": "tokenizers",
"bos_token": "<bos>",
"eos_token": "<eos>",
"model_max_length": 1000000000000000019884624838656,
"pad_token": "<pad>",
"tokenizer_class": "GPT2Tokenizer"
}

Xet Storage Details

Size:
195 Bytes
·
Xet hash:
d4d7384efae402f9af625b511a857d049b937aed4ba436adaac48c29dc89fceb

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.