theprint commited on
Commit
602ae0c
·
verified ·
1 Parent(s): 44658b4

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +61 -0
README.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/Qwen3.5-2B
3
+ tags:
4
+ - fine-tuned
5
+ - lora
6
+ - sft
7
+ - auto-sft
8
+ language:
9
+ - en
10
+ library_name: transformers
11
+ ---
12
+
13
+ # ComputerScience-v1-2B
14
+
15
+ A fine-tuned version of [`unsloth/Qwen3.5-2B`](https://huggingface.co/unsloth/Qwen3.5-2B) trained on **ComputerScience ML AI Chatml 7.0k 110926 train** data using [Auto-SFT](https://github.com/theprint/auto-sft) — an automated hyperparameter search and supervised fine-tuning pipeline.
16
+
17
+ The base model was adapted to follow the style and content of the `ComputerScience ML AI Chatml 7.0k 110926 train` dataset. Expect improved performance on tasks similar to those represented in the training data.
18
+
19
+ ## Model Details
20
+
21
+ | Property | Value |
22
+ |---|---|
23
+ | Base model | `unsloth/Qwen3.5-2B` |
24
+ | Training data | `data/ComputerScience-ML-AI-Chatml-7.0k-110926_train.json` |
25
+ | Fine-tuning epochs | 2 |
26
+ | Fine-tuning date | 2026-09-12 |
27
+ | Fine-tuning method | LoRA (merged to full 16-bit) |
28
+
29
+ ## Training Hyperparameters
30
+
31
+ ### LoRA
32
+
33
+ | Parameter | Value |
34
+ |---|---|
35
+ | `r` | `4` |
36
+ | `alpha` | `8` |
37
+ | `dropout` | `0.02` |
38
+ | `target_modules` | `['q_proj', 'v_proj', 'k_proj', 'o_proj', 'gate_proj', 'up_proj', 'down_proj']` |
39
+
40
+ ### Training
41
+
42
+ | Parameter | Value |
43
+ |---|---|
44
+ | `learning_rate` | `0.0005` |
45
+ | `batch_size` | `4` |
46
+ | `gradient_accumulation_steps` | `1` |
47
+ | `warmup_ratio` | `0.0` |
48
+ | `max_seq_length` | `2048` |
49
+ | `quantization` | `none` |
50
+
51
+ ## Usage
52
+
53
+ ```python
54
+ from transformers import AutoModelForCausalLM, AutoTokenizer
55
+
56
+ model = AutoModelForCausalLM.from_pretrained("theprint/ComputerScience-v1-2B")
57
+ tokenizer = AutoTokenizer.from_pretrained("theprint/ComputerScience-v1-2B")
58
+ ```
59
+
60
+ ---
61
+ *Generated by [Auto-SFT](https://github.com/theprint/auto-sft) — automated LoRA fine-tuning with hyperparameter search.*