mratsim

/

GLM-4.7-EXL3

like 12

It's not a secret, maybe I'll add the overrides to the repo, here is how to reproduce the 2.10bpw specifically:

# Base 2-bit
sources:
  - id: 3
    model_dir: [...]/glm-4.7-exl3-3bpw
  - id: 4
    model_dir: [...]/glm-4.7-exl3-4bpw
  - id: 6
    model_dir: [...]/glm-4.7-exl3-6bpw
  - id: 16
    model_dir: [HF_HUB]/models--zai-org--GLM-4.7/snapshots/9fcd12a1f7dcaa3b866d5140498eba03eed5529d
overrides: # last to match applies
  - key: "model.layers.*.self_attn.q_proj.*"
    source: 3
  - key: "model.layers.*.self_attn.k_proj.*"
    source: 4
  - key: "model.layers.*.self_attn.v_proj.*"
    source: 4
  - key: "model.layers.*.self_attn.o_proj.*"
    source: 4

  - key: "model.embed_tokens.*"
    source: 16
  - key: "model.norm.*"
    source: 16
#  - key: "lm_head.*"
#    source: 2 # 6-bit
  - key: "model.layers.*.mlp.gate.*" # MoE router
    source: 6
  - key: "model.layers.*.mlp.shared_experts.*"
    source: 6
  - key: "model.layers.*.input_layernorm.*"
    source: 6
  - key: "model.layers.*.post_attention_layernorm.*"
    source: 6
  - key: "model.layers.0.*"
    source: 4
  - key: "model.layers.1.*"
    source: 4
  - key: "model.layers.2.*"
    source: 4
  - key: "model.layers.90.*.down_proj"
    source: 4
  - key: "model.layers.91.*.down_proj"
    source: 4

It's not a secret, maybe I'll add the overrides to the repo, here is how to reproduce the 2.10bpw specifically:

# Base 2-bit
sources:
  - id: 3
    model_dir: [...]/glm-4.7-exl3-3bpw
  - id: 4
    model_dir: [...]/glm-4.7-exl3-4bpw
  - id: 6
    model_dir: [...]/glm-4.7-exl3-6bpw
  - id: 16
    model_dir: [HF_HUB]/models--zai-org--GLM-4.7/snapshots/9fcd12a1f7dcaa3b866d5140498eba03eed5529d
overrides: # last to match applies
  - key: "model.layers.*.self_attn.q_proj.*"
    source: 3
  - key: "model.layers.*.self_attn.k_proj.*"
    source: 4
  - key: "model.layers.*.self_attn.v_proj.*"
    source: 4
  - key: "model.layers.*.self_attn.o_proj.*"
    source: 4

  - key: "model.embed_tokens.*"
    source: 16
  - key: "model.norm.*"
    source: 16
#  - key: "lm_head.*"
#    source: 2 # 6-bit
  - key: "model.layers.*.mlp.gate.*" # MoE router
    source: 6
  - key: "model.layers.*.mlp.shared_experts.*"
    source: 6
  - key: "model.layers.*.input_layernorm.*"
    source: 6
  - key: "model.layers.*.post_attention_layernorm.*"
    source: 6
  - key: "model.layers.0.*"
    source: 4
  - key: "model.layers.1.*"
    source: 4
  - key: "model.layers.2.*"
    source: 4
  - key: "model.layers.90.*.down_proj"
    source: 4
  - key: "model.layers.91.*.down_proj"
    source: 4

thanks for sharing this and let me figure out how to use this config 🤤

Look this 3D City Scene with one shot generation. Prompt: Create a 3D city scene using Three.js that features a bustling urban environment with skyscrapers, apartment buildings, and smaller shops lining the streets. Incorporate roads with moving cars, traffic lights, and pedestrian crossings to bring the city to life. Add pedestrians walking on sidewalks and crossing the streets to enhance realism. Include street elements such as lampposts, benches, and trees for a more immersive experience. Utilize dynamic lighting to simulate day and night cycles, and implement basic camera controls to allow users to explore the vibrant cityscape from different perspectives.

Whoa, impressive!