Text Generation
Transformers
PyTorch
Safetensors
English
gpt2
idk
english
text
langauge
speaking
text-generation-inference
How to use from
SGLangUse Docker images
docker run --gpus all \
--shm-size 32g \
-p 30000:30000 \
-v ~/.cache/huggingface:/root/.cache/huggingface \
--env "HF_TOKEN=<secret>" \
--ipc=host \
lmsysorg/sglang:latest \
python3 -m sglang.launch_server \
--model-path "Roman190928/MicroGPT" \
--host 0.0.0.0 \
--port 30000# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "Roman190928/MicroGPT",
"prompt": "Once upon a time,",
"max_tokens": 512,
"temperature": 0.5
}'Quick Links
Tbh i didn't Eval the model, im not sure what it can do honestly
Feel free to do whatever you want with it, however, it's just been trained
Total parameters: 51,475,968
Trained on 1 T4 in like... 1 hour
2.7 GB vram used when training
... uhm
So it runs in like... 5 seconds on a T4
but it kinda... types like this
Prompt : yo microgpt what's good?
response V
[/INST] I hope we can be very hard to be a bit when it? [/INST] I wish we can help us feel that you think about how people can be a great plan. [/INST] I agree that we feel a small things, it is important for everyone feel more. [/INST] I just want to stay positive about how my spirits and enjoy it feels like it seems like a bit sad [/INST] Yes, don't it is amazing how we have to feel quite disgusted
ill make another one... that hopefully doesnt babble like this... soonish...
Edit
Thanks for the downloads guys! 😀
- Downloads last month
- 4
Install from pip and serve model
# Install SGLang from pip: pip install sglang# Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Roman190928/MicroGPT" \ --host 0.0.0.0 \ --port 30000# Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Roman190928/MicroGPT", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'