mradermacher

/

model_requests

like 175

We have 10 petabytes of quants... we for sure dont retain them. Perhaps let's ask @DavidAU himself

You could convert the Q8_0 GGUF back into the original model using https://github.com/purinnohito/gguf_to_safetensors as the loss in quality is very minimal but the process is a bit of a pain.

Due to the rapid development pace of llama.cpp and the GGUF format, it appears that recent GGUF formats cannot be correctly converted to safetensors. As it is difficult to keep up with these changes, I have decided to end development at its current state.

So you will need to fix the resulting Safetensor model using my script:

import os
import shutil
from safetensors import safe_open
from safetensors.torch import save_file
from concurrent.futures import ThreadPoolExecutor, as_completed

SOURCE_DIR = "/nvme/Huihui-MiniMax-M2-abliterated"
TARGET_DIR = "/nvme/Huihui-MiniMax-M2-abliterated-repaired"
NUM_THREADS = 3

os.makedirs(TARGET_DIR, exist_ok=True)

# Copy config/tokenizer/custom code files
for fname in os.listdir(SOURCE_DIR):
    src = os.path.join(SOURCE_DIR, fname)
    dst = os.path.join(TARGET_DIR, fname)

    if fname.startswith("model-") and fname.endswith(".safetensors"):
        continue  # shards handled separately

    if os.path.isfile(src):
        shutil.copy2(src, dst)

print("✓ Copied config/tokenizer files")

# Find shards
shards = sorted([
    f for f in os.listdir(SOURCE_DIR)
    if f.startswith("model-") and f.endswith(".safetensors")
])

print(f"Found {len(shards)} shards total\n")

def repair_shard(shard):
    src_path = os.path.join(SOURCE_DIR, shard)
    dst_path = os.path.join(TARGET_DIR, shard)

    # Skip if already repaired
    if os.path.exists(dst_path):
        return f"skipped {shard}"

    tensors = {}
    with safe_open(src_path, framework="pt") as f:
        for key in f.keys():
            tensors[key] = f.get_tensor(key)

    save_file(tensors, dst_path)
    return f"repaired {shard}"

# Run repairs concurrently
with ThreadPoolExecutor(max_workers=NUM_THREADS) as executor:
    futures = {executor.submit(repair_shard, shard): shard for shard in shards}

    for future in as_completed(futures):
        result = future.result()
        print("✓", result)

print("\nAll shards processed. Output stored in:")
print(TARGET_DIR)

Source is up ; that is a neat script BTW.