File size: 4,116 Bytes
cdcea68 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 |
---
language: en
license: apache-2.0
tags:
- text-classification
- multi-domain
- phi-3
- lora
- domain-classification
datasets:
- custom
metrics:
- accuracy
- f1
library_name: transformers
pipeline_tag: text-classification
---
# Multi-Domain Classifier (Phi-3)
A fine-tuned domain classification model based on Microsoft's Phi-3-mini-4k-instruct, trained to classify queries into 17 domains and detect multi-domain queries.
## Model Description
This model classifies text queries into domains and identifies when queries span multiple domains. It returns structured JSON output with primary domain, confidence scores, multi-domain flag, and secondary domains.
### Supported Domains (17 total)
ambiguous, api_generation, business, coding, creative_content, data_analysis, education, general_knowledge, geography, history, law, literature, mathematics, medicine, science, sensitive, technology
## Training Details
- **Base Model**: microsoft/Phi-3-mini-4k-instruct
- **Training Method**: LoRA (Low-Rank Adaptation)
- **Training Samples**: 3,666
- **Multi-Domain Samples**: 516 (14.1%)
- **Training Time**: 1.61 hours
- **LoRA Rank**: 32
- **LoRA Alpha**: 64
### Training Configuration
```python
{
"num_epochs": 5,
"batch_size": 16,
"learning_rate": 0.0002,
"warmup_ratio": 0.1,
"weight_decay": 0.01,
"gradient_accumulation_steps": 2,
"eval_steps": 50,
"save_steps": 100,
"logging_steps": 10
}
```
## Performance
- **Primary Domain Accuracy**: 97.25%
- **F1 Score (Macro)**: 0.9177458758572196
- **Multi-Domain Detection F1**: 0.9523809523809523
## Usage
### Installation
```bash
pip install transformers torch peft
```
### Basic Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
import json
# Load model
base_model = AutoModelForCausalLM.from_pretrained(
"microsoft/Phi-3-mini-4k-instruct",
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "ovinduG/multi-domain-classifier-phi3")
tokenizer = AutoTokenizer.from_pretrained("ovinduG/multi-domain-classifier-phi3")
# Prepare input
query = "Build a machine learning model to analyze sales data"
prompt = f'''Classify this query: {query}
Output JSON format:
{
"primary_domain": "domain_name",
"primary_confidence": 0.95,
"is_multi_domain": true/false,
"secondary_domains": [{"domain": "name", "confidence": 0.85}]
}'''
# Generate
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.1)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
# Parse result
result = json.loads(response.split("Output JSON format:")[-1].strip())
print(result)
```
### Output Format
```json
{
"primary_domain": "data_analysis",
"primary_confidence": 0.92,
"is_multi_domain": true,
"secondary_domains": [
{"domain": "machine_learning", "confidence": 0.85},
{"domain": "business", "confidence": 0.72}
]
}
```
## Use Cases
- **Query Routing**: Route user queries to specialized models or APIs based on domain
- **Intent Classification**: Understand the domain and complexity of user requests
- **Multi-Domain Detection**: Identify queries that require expertise in multiple areas
- **Content Categorization**: Classify documents and articles by domain
## Limitations
- Model may struggle with very rare domains (< 50 training examples)
- Performance may degrade on domains not seen during training
- Confidence scores are relative and may need calibration for production use
- Secondary domain detection accuracy varies based on query clarity
## Training Data
Trained on a custom dataset of 3,666 queries across 17 domains, with 14.1% multi-domain examples.
## Citation
```bibtex
@misc{multi-domain-classifier-phi3,
author = {ovinduG},
title = {Multi-Domain Classifier based on Phi-3},
year = {2024},
publisher = {HuggingFace},
url = {https://huggingface.co/ovinduG/multi-domain-classifier-phi3}
}
```
## Model Card Authors
- ovinduG
## License
Apache 2.0 (following base model license)
|