Spaces:
Sleeping
Sleeping
File size: 2,045 Bytes
f62dcda de17cfa f62dcda 4bb337f f62dcda 5f1ec06 f62dcda bc96a2a f62dcda bc96a2a f62dcda bc96a2a f62dcda a178e35 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
title: Qwen Training
emoji: 🧙
colorFrom: purple
colorTo: blue
sdk: gradio
sdk_version: 5.49.1
app_file: app.py
pinned: false
license: mit
suggested_hardware: zero-a10g
short_description: app.py is base workkng
---
# PromptWizard Qwen Fine-tuning
This Space fine-tunes Qwen models using Gita dataset wit optimization methodology.
## Features
- **GPU-Accelerated Training**: Uses HuggingFace's GPU infrastructure for fast training
- **LoRA Fine-tuning**: Efficient parameter-efficient fine-tuning
- **GITA Dataset**: High-quality use any custiom reasoning dataset
- **PromptWizard Integration**: Uses Microsoft's PromptWizard evaluation methodology
- **Auto Push to Hub**: Trained models are automatically uploaded to HuggingFace Hub
## How to Use
1. Select your base model (default: Qwen/Qwen2.5-7B)
2. Configure training parameters:
- Number of epochs (3-5 recommended)
- Batch size (4-8 for T4 GPU)
- Learning rate (2e-5 is a good default)
3. Click "Start Training" and monitor the output
4. The trained model will be pushed to HuggingFace Hub
## Training Data
The Space uses the GITA dataset, which contains grade school math problems. The data is formatted according to PromptWizard specifications for optimal prompt optimization.
## Model Output
After training, the model will be available at:
- HuggingFace Hub: `your-username/promptwizard-qwen-gsm8k`
- Local download: Available in the Space's output directory
## Technical Details
- **Base Model**: Qwen2.5-7B (or your choice)
- **Training Method**: LoRA with rank 16
- **Quantization**: 8-bit for memory efficiency
- **Mixed Precision**: FP16 for faster training
- **Gradient Checkpointing**: Enabled for memory savings
## Resource Requirements
- **GPU**: T4 or better recommended
- **Memory**: 16GB+ GPU memory
- **Training Time**: ~30-60 minutes on T4
## Citation
If you use this training setup, please cite:
```bibtex
@misc{promptwizard2024,
title={PromptWizard: Task-Aware Prompt Optimization},
author={Microsoft Research},
year={2024}
}
``` |