metadata
license: mit
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- testing
- random-initialization
- gpt2
Tiny random GPT-2 for testing
This model is randomly initialized and has not been trained. It is for testing upload, download, tokenization, and model loading only. Its output is not meaningful and it is not suitable for real language tasks or benchmarking. No pretrained model weights or training datasets were used.
Architecture: 1 GPT-2 layer, 1 attention head, 16 hidden dimensions, 32 vocabulary tokens, and a maximum context length of 64 tokens. Parameter count: 3792.
Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "ruhook/test-ruhook"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
inputs = tokenizer("hello world", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=5, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Validated locally with Python 3, torch 2.2.2 and transformers 4.46.3. The toy word-level tokenizer maps words outside its small vocabulary to UNK.