llhax/qwen-dialog-to-json

Qwen2.5-3B-Instruct Dialog-to-JSON

QLoRA fine-tuned on DialogSum to extract structured JSON from raw conversational dialogues.

Model Description

This model takes a raw dialogue as input and outputs a structured JSON object with topic and summary keys. It is built for use cases that require converting unstructured conversational data into machine-readable format for downstream pipelines, databases, or APIs.

Training Details

Base model	Qwen/Qwen2.5-3B-Instruct
Method	QLoRA — 4-bit NF4 quantization + LoRA
LoRA rank	r=64, alpha=64
Target modules	q_proj, k_proj, v_proj, o_proj
Dataset	DialogSum (12,460 examples)
Epochs	1
Learning rate	2e-4
Optimizer	paged_adamw_32bit
Hardware	Kaggle P100 (16GB)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch, json

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained("llhax/qwen-dialog-to-json")

model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-3B-Instruct",
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16
)
model = PeftModel.from_pretrained(model, "llhax/qwen-dialog-to-json")
model.eval()

SYSTEM_PROMPT = (
    "You are a helpful assistant that extracts dialogue into a JSON object "
    "with 'topic' and 'summary' keys."
)

def extract_json(dialogue: str) -> dict:
    prompt = tokenizer.apply_chat_template(
        [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user",   "content": dialogue}
        ],
        tokenize=False,
        add_generation_prompt=True
    )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=128,
            do_sample=False,
            repetition_penalty=1.1
        )
    raw = tokenizer.decode(
        outputs[0][inputs["input_ids"].shape[1]:],
        skip_special_tokens=True
    )
    return json.loads(raw)

Example Output

Input:

Alex: The project deadline is tomorrow.
Lisa: Is everything ready?
Alex: Almost, just need to finish the report.
Lisa: I can help you with that tonight.

Output:

{
  "topic": "project deadline",
  "summary": "Alex and Lisa discuss the upcoming deadline. Lisa offers to help finish the report tonight."
}

Limitations

• 1 epoch training, additional epochs would improve consistency further

Demo

HuggingFace Space