---
license: mit
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- testing
- random-initialization
- gpt2
---
# Tiny random GPT-2 for testing

This model is randomly initialized and **has not been trained**. It is for
testing upload, download, tokenization, and model loading only. Its output is
not meaningful and it is not suitable for real language tasks or benchmarking.
No pretrained model weights or training datasets were used.

Architecture: 1 GPT-2 layer, 1 attention head, 16 hidden dimensions,
32 vocabulary tokens, and a maximum context length of 64 tokens.
Parameter count: 3792.

## Usage
```python
from transformers import AutoTokenizer, AutoModelForCausalLM

repo = "ruhook/test-ruhook"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
inputs = tokenizer("hello world", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=5, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

Validated locally with Python 3, torch 2.2.2 and transformers 4.46.3.
The toy word-level tokenizer maps words outside its small vocabulary to UNK.