Instructions to use claritylab/zero-shot-implicit-gpt2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use claritylab/zero-shot-implicit-gpt2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="claritylab/zero-shot-implicit-gpt2")

# Load model directly
from transformers import AutoTokenizer, AutoModelWithLMHead

tokenizer = AutoTokenizer.from_pretrained("claritylab/zero-shot-implicit-gpt2")
model = AutoModelWithLMHead.from_pretrained("claritylab/zero-shot-implicit-gpt2")

sentence-transformers

How to use claritylab/zero-shot-implicit-gpt2 with sentence-transformers:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("claritylab/zero-shot-implicit-gpt2")

sentences = [
    "The weather is lovely today.",
    "It's so sunny outside!",
    "He drove to the stadium."
]
embeddings = model.encode(sentences)

similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use claritylab/zero-shot-implicit-gpt2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "claritylab/zero-shot-implicit-gpt2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claritylab/zero-shot-implicit-gpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/claritylab/zero-shot-implicit-gpt2

SGLang

How to use claritylab/zero-shot-implicit-gpt2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "claritylab/zero-shot-implicit-gpt2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claritylab/zero-shot-implicit-gpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "claritylab/zero-shot-implicit-gpt2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "claritylab/zero-shot-implicit-gpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use claritylab/zero-shot-implicit-gpt2 with Docker Model Runner:
```
docker model run hf.co/claritylab/zero-shot-implicit-gpt2
```

zero-shot-implicit-gpt2 / README.md

StefanH

Update README.md

f574d69 about 3 years ago

preview code

raw

history blame contribute delete

2.48 kB

	---
	library_name: zeroshot_classifier
	tags:
	- transformers
	- sentence-transformers
	- zeroshot_classifier
	license: mit
	datasets:
	- claritylab/UTCD
	language:
	- en
	pipeline_tag: text-generation
	metrics:
	- accuracy
	---

	# Zero-shot Implicit GPT2

	This is a modified GPT2 model.
	It was introduced in the Findings of ACL'23 Paper Label Agnostic Pre-training for Zero-shot Text Classification by *Christopher Clarke, Yuzhao Heng, Yiping Kang, Krisztian Flautner, Lingjia Tang and Jason Mars*.
	The code for training and evaluating this model can be found [here](https://github.com/ChrisIsKing/zero-shot-text-classification/tree/master).

	## Model description

	This model is intended for zero-shot text classification.
	It was trained under the generative classification framework via implicit training with the aspect-normalized [UTCD](https://huggingface.co/datasets/claritylab/UTCD) dataset.

	- Finetuned from model: [`gpt2-medium`](https://huggingface.co/gpt2-medium)


	## Usage

	Install our [python package](https://pypi.org/project/zeroshot-classifier/):
	```bash
	pip install zeroshot-classifier
	```

	Then, you can use the model like this:

	```python
	>>> import torch
	>>> from zeroshot_classifier.models import ZsGPT2Tokenizer, ZsGPT2LMHeadModel

	>>> training_strategy = 'implicit'
	>>> model_name = f'claritylab/zero-shot-{training_strategy}-gpt2'
	>>> model = ZsGPT2LMHeadModel.from_pretrained(model_name)
	>>> tokenizer = ZsGPT2Tokenizer.from_pretrained(model_name, form=training_strategy)

	>>> text = "I'd like to have this track onto my Classical Relaxations playlist."
	>>> labels = [
	>>> 'Add To Playlist', 'Book Restaurant', 'Get Weather', 'Play Music', 'Rate Book', 'Search Creative Work',
	>>> 'Search Screening Event'
	>>> ]
	>>> aspect = 'intent'

	>>> inputs = tokenizer(dict(text=text, label_options=labels, aspect=aspect), mode='inference-sample')
	>>> inputs = {k: torch.tensor(v).unsqueeze(0) for k, v in inputs.items()}
	>>> outputs = model.generate(**inputs, max_length=128)
	>>> decoded = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
	>>> print(decoded)

	<\|question\|>Which of these categories best describes the following document? : " Play Music ", " Add To Playlist ", " Rate Book ", " Book Restaurant ", " Search Creative Work ", " Search Screening Event ", " Get Weather "<\|endoftext\|><\|text\|>intent[ASPECT_SEP]I'd like to have this track onto my Classical Relaxations playlist.<\|endoftext\|><\|answer\|>Play Media<\|endoftext\|>
	```