Menlo

/

Lucy

like 64

Follow

Menlo Research 654

When using the transformers and running the model, the generate function does not work as it assumes it is base model not instruct. How do you handle this?
TypeError: The current model class (Qwen3Model) is not compatible with .generate(), as it doesn't have a language model head. Classes that support generation often end in one of these names: ['ForCausalLM', 'ForConditionalGeneration', 'ForSpeechSeq2Seq', 'ForVision2Seq'].

# Load model directly
from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("Menlo/Lucy")
tokenizer = AutoTokenizer.from_pretrained("Menlo/Lucy")

messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)