QwQ-SuperNatural-3B

QwQ-SuperNatural-3B is a Qwen2.5-based supernatural model designed to provide context-based supernatural responses from the input it receives. It has 3 billion parameters and is a domain-specific, supervised fine-tuned model. The model demonstrates significant improvements in instruction following, generating long texts (over 8K tokens), understanding structured data (e.g., tables), and generating structured outputs, especially in JSON format. It is also more resilient to the diversity of system prompts, enhancing role-play implementation and condition-setting for chatbots.

SuperNatural Colab Demo

Notebook	Description	Link
Colab Demo	Interactive demo for the QwQ-SuperNatural-3B model using Google Colab.	Open in Colab

Quickstart with Transformers

Here provides a code snippet with apply_chat_template to show you how to load the tokenizer and model and how to generate contents.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "prithivMLmods/QwQ-SuperNatural-3B"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "system", "content": "You are an Super Natural Bot, You are a helpful assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

Intended Use:

QwQ-SuperNatural-3B is designed for:

1. Role-play and interactive chatbots: It excels in generating contextually relevant and engaging supernatural-themed responses.
2. Long-form content generation: Its capability to handle over 8,000 tokens makes it suitable for generating detailed narratives, articles, or creative writing.
3. Structured data understanding: The model can process and interpret structured inputs such as tables, schemas, and JSON formats, making it useful for data-driven applications.
4. Dynamic prompt responses: Its resilience to diverse prompts makes it ideal for applications requiring adaptable behavior, such as virtual assistants and domain-specific simulations.

Limitations:

1. Domain specificity: While fine-tuned for supernatural contexts, its general knowledge might be less accurate or nuanced outside this domain.
2. Token constraints: Although capable of generating long texts, extremely large inputs or outputs might exceed processing limits.
3. Bias and creativity trade-offs: The model may reflect biases present in its training data and could produce less creative or diverse outputs in domains where it lacks fine-tuning.
4. Reliance on input clarity: Ambiguous or poorly structured prompts can lead to less coherent or contextually accurate responses.
5. Computational requirements: Handling a model with 3 billion parameters requires significant computational resources, which may limit its accessibility for smaller-scale applications.