Update README.md
Browse filesAdded links to more quants
README.md
CHANGED
|
@@ -121,9 +121,15 @@ If you save this as a .json file, you can import it directly into Silly Tavern.
|
|
| 121 |
This repo contains branches for various exllama2 quanizations of the model calibratend on a version of the PIPPA dataset.
|
| 122 |
|
| 123 |
* Main Branch, Full weights
|
| 124 |
-
* 3.2 bpw -- This will fit comfortably within 48 GB of VRAM at 8192 context.
|
| 125 |
-
* 3.35 bpw
|
| 126 |
-
* 3.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 127 |
|
| 128 |
### Licence and usage restrictions
|
| 129 |
|
|
|
|
| 121 |
This repo contains branches for various exllama2 quanizations of the model calibratend on a version of the PIPPA dataset.
|
| 122 |
|
| 123 |
* Main Branch, Full weights
|
| 124 |
+
* 3.2 bpw branch -- This will fit comfortably within 48 GB of VRAM at 8192 context.
|
| 125 |
+
* 3.35 bpw branch -- This will fit within 48 GB of VRAM at 4096 context without using the 8-bit cache setting.
|
| 126 |
+
* 3.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-3.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-3.0bpw-h6-exl2-2)
|
| 127 |
+
* 4.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-4.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-4.0bpw-h6-exl2-2)
|
| 128 |
+
* 5.0 bpw -- [LoneStriker/Rogue-Rose-103b-v0.2-5.0bpw-h6-exl2-2](https://huggingface.co/LoneStriker/Rogue-Rose-103b-v0.2-5.0bpw-h6-exl2-2)
|
| 129 |
+
* GGUF -- [TheBloke/Rogue-Rose-103b-v0.2-GGUF](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-GGUF)
|
| 130 |
+
* AWQ -- [TheBloke/Rogue-Rose-103b-v0.2-AWQ](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-AWQ)
|
| 131 |
+
* GPTQ -- [TheBloke/Rogue-Rose-103b-v0.2-GPTQ](https://huggingface.co/TheBloke/Rogue-Rose-103b-v0.2-GPTQ)
|
| 132 |
+
|
| 133 |
|
| 134 |
### Licence and usage restrictions
|
| 135 |
|