README.md

metadata

language: en
license: apache-2.0
tags:
  - text-classification
  - multi-domain
  - phi-3
  - lora
  - domain-classification
datasets:
  - custom
metrics:
  - accuracy
  - f1
library_name: transformers
pipeline_tag: text-classification

Multi-Domain Classifier (Phi-3)

A fine-tuned domain classification model based on Microsoft's Phi-3-mini-4k-instruct, trained to classify queries into 17 domains and detect multi-domain queries.

Model Description

This model classifies text queries into domains and identifies when queries span multiple domains. It returns structured JSON output with primary domain, confidence scores, multi-domain flag, and secondary domains.

Supported Domains (17 total)

ambiguous, api_generation, business, coding, creative_content, data_analysis, education, general_knowledge, geography, history, law, literature, mathematics, medicine, science, sensitive, technology

Training Details

• Base Model: microsoft/Phi-3-mini-4k-instruct
• Training Method: LoRA (Low-Rank Adaptation)
• Training Samples: 3,666
• Multi-Domain Samples: 516 (14.1%)
• Training Time: 1.61 hours
• LoRA Rank: 32
• LoRA Alpha: 64

Training Configuration

{
  "num_epochs": 5,
  "batch_size": 16,
  "learning_rate": 0.0002,
  "warmup_ratio": 0.1,
  "weight_decay": 0.01,
  "gradient_accumulation_steps": 2,
  "eval_steps": 50,
  "save_steps": 100,
  "logging_steps": 10
}

Performance

• Primary Domain Accuracy: 97.25%
• F1 Score (Macro): 0.9177458758572196
• Multi-Domain Detection F1: 0.9523809523809523

Usage

Installation

pip install transformers torch peft

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
import json

# Load model
base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

model = PeftModel.from_pretrained(base_model, "ovinduG/multi-domain-classifier-phi3")
tokenizer = AutoTokenizer.from_pretrained("ovinduG/multi-domain-classifier-phi3")

# Prepare input
query = "Build a machine learning model to analyze sales data"

prompt = f'''Classify this query: {query}

Output JSON format:
{
  "primary_domain": "domain_name",
  "primary_confidence": 0.95,
  "is_multi_domain": true/false,
  "secondary_domains": [{"domain": "name", "confidence": 0.85}]
}'''

# Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Parse result
result = json.loads(response.split("Output JSON format:")[-1].strip())
print(result)

Output Format

{
  "primary_domain": "data_analysis",
  "primary_confidence": 0.92,
  "is_multi_domain": true,
  "secondary_domains": [
    {"domain": "machine_learning", "confidence": 0.85},
    {"domain": "business", "confidence": 0.72}
  ]
}

Use Cases

• Query Routing: Route user queries to specialized models or APIs based on domain
• Intent Classification: Understand the domain and complexity of user requests
• Multi-Domain Detection: Identify queries that require expertise in multiple areas
• Content Categorization: Classify documents and articles by domain

Limitations

• Model may struggle with very rare domains (< 50 training examples)
• Performance may degrade on domains not seen during training
• Confidence scores are relative and may need calibration for production use
• Secondary domain detection accuracy varies based on query clarity

Training Data

Trained on a custom dataset of 3,666 queries across 17 domains, with 14.1% multi-domain examples.

Citation

@misc{multi-domain-classifier-phi3,
  author = {ovinduG},
  title = {Multi-Domain Classifier based on Phi-3},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/ovinduG/multi-domain-classifier-phi3}
}

Model Card Authors

• ovinduG

License

Apache 2.0 (following base model license)