diff --git a/README.md b/README.md new file mode 100644 index 0000000000000000000000000000000000000000..0aeb1d4f22fb89bedfbc746e70c0649485c22dec --- /dev/null +++ b/README.md @@ -0,0 +1,224 @@ +--- +language: +- en +library_name: transformers +tags: +- glm +- MOE +- pruning +- compression +license: mit +name: cerebras/GLM-4.6-REAP-218B-A32B-FP8 +description: > + This model was obtained by uniformly pruning 40% of experts in GLM-4.6-FP8 using the REAP method. +readme: > + https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B-FP8/main/README.md +license_link: https://huggingface.co/zai-org/GLM-4.6-FP8/blob/main/LICENSE +pipeline_tag: text-generation +base_model: +- zai-org/GLM-4.6-FP8 +--- + +

+ ๐“Œณ REAP๐“Œณ the Experts: Why Pruning Prevails for One-Shot MoE Compression
+ REAP +

+ +# GLM-4.6-REAP-218B-A32B-FP8 + +## โœจ Highlights + +Introducing **GLM-4.6-REAP-218B-A32B-FP8**, a **memory-efficient compressed variant** of GLM-4.6-FP8 that maintains near-identical performance while being **40% lighter**. + +This model was created using **REAP (Router-weighted Expert Activation Pruning)**, a novel expert pruning method that selectively removes redundant experts while preserving the router's independent control over remaining experts. Key features include: + +- **Near-Lossless Performance**: Maintains almost identical accuracy on code generation, agentic coding, and function calling tasks compared to the full 355B model +- **40% Memory Reduction**: Compressed from 355B to 218B parameters, significantly lowering deployment costs and memory requirements +- **Preserved Capabilities**: Retains all core functionalities including code generation, agentic workflows, repository-scale understanding, and function calling +- **Drop-in Compatibility**: Works with vanilla vLLM - no source modifications or custom patches required +- **Optimized for Real-World Use**: Particularly effective for resource-constrained environments, local deployments, and academic research + +**For downstream low-bit quantization, we suggest using the [BF16 variant](https://huggingface.co/cerebras/GLM-4.6-REAP-218B-A32B).** + +--- +## ๐Ÿ“‹ Model Overview + +**GLM-4.6-REAP-218B-A32B-FP8** has the following specifications: + +- **Base Model**: GLM-4.6-FP8 +- **Compression Method**: REAP (Router-weighted Expert Activation Pruning) +- **Compression Ratio**: 40% expert pruning +- **Type**: Sparse Mixture-of-Experts (SMoE) Causal Language Model +- **Number of Parameters**: 218B total, 32B activated per token +- **Number of Layers**: 92 +- **Number of Attention Heads (GQA)**: 96 for Q and 8 for KV +- **Number of Experts**: 96 (uniformly pruned from 160) +- **Number of Activated Experts**: 8 per token +- **Context Length**: 202,752 tokens +- **License**: MIT + +--- + +## ๐Ÿ“Š Evaluations + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
BenchmarkGLM-4.6-FP8GLM-4.6-REAP-268B-A32B-FP8GLM-4.6-REAP-252B-A32B-FP8GLM-4.6-REAP-218B-A32B-FP8
Compressionโ€”25%30%40%
Coding
HumanEval96.396.395.795.1
HumanEval+93.391.590.990.2
MBPP87.689.989.989.4
MBPP+73.574.973.573.8
Reasoning
GPQA diamond (thinking)78.876.875.869.7
AIME25 (thinking)90.093.390.090.0
MATH-500 (thinking)95.597.094.893.3
Tool Calling
BFCL-v3 (thinking)78.477.376.874.2
+ + +๐ŸŸฉ *This checkpoint maintains almost identical performance while being 40% lighter.* + +For more details on the evaluation setup, refer to the [REAP arXiv preprint](https://arxiv.org/abs/2510.13999). + +--- + +## ๐Ÿš€ Deployment + +You can deploy the model directly using the **latest vLLM** (v0.11.0), no source modifications or custom patches required. + +```bash +vllm serve cerebras/GLM-4.6-REAP-218B-A32B-FP8 \ + --tensor-parallel-size 8 \ + --tool-call-parser glm45 \ + --enable-auto-tool-choice \ + --enable-expert-parallel +``` + +If you encounter insufficient memory when running this model, you might need to set a lower value for `--max-num-seqs` flag (e.g. set to 64). + + +## ๐Ÿงฉ Model Creation + +This checkpoint was created by applying the **REAP (Router-weighted Expert Activation Pruning)** method uniformly across all Mixture-of-Experts (MoE) blocks of **GLM-4.6-FP8**, with a **40% pruning rate**. + +### How REAP Works + +REAP selects experts to prune based on a novel **saliency criterion** that considers both: +- **Router gate values**: How frequently and strongly the router activates each expert +- **Expert activation norms**: The magnitude of each expert's output contributions + +This dual consideration ensures that experts contributing minimally to the layer's output are pruned, while preserving those that play critical roles in the model's computations. + +### Key Advantages + +- **One-Shot Compression**: No fine-tuning required after pruning - the model is immediately ready for deployment +- **Preserved Router Control**: Unlike expert merging methods, REAP maintains the router's independent, input-dependent control over remaining experts, avoiding "functional subspace collapse" +- **Generative Task Superiority**: REAP significantly outperforms expert merging approaches on generative benchmarks (code generation, creative writing, mathematical reasoning) while maintaining competitive performance on discriminative tasks + +### Calibration + +The model was calibrated using a diverse mixture of domain-specific datasets including: +- Code generation samples ([evol-codealpaca](https://huggingface.co/datasets/theblackcat102/evol-codealpaca-v1)) +- Function calling examples ([xlam-function-calling](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k)) +- Agentic multi-turn trajectories ([SWE-smith-trajectories](https://huggingface.co/datasets/SWE-bench/SWE-smith-trajectories)) + +๐Ÿ“š For more details, refer to the following resources: + +- [๐Ÿงพ arXiv Preprint](https://arxiv.org/abs/2510.13999) +- [๐Ÿงพ REAP Blog](https://www.cerebras.ai/blog/reap) +- [๐Ÿ’ป REAP Codebase (GitHub)](https://github.com/CerebrasResearch/reap) + +--- + +## โš–๏ธ License + +This model is derived from +**[`zai-org/GLM-4.6-FP8`](https://huggingface.co/zai-org/GLM-4.6-FP8)** +and distributed under the **MIT license**. + +--- + +## ๐Ÿงพ Citation + +If you use this checkpoint, please cite the REAP paper: + +```bibtex +@article{lasby-reap, + title={REAP the Experts: Why Pruning Prevails for One-Shot MoE compression}, + author={Lasby, Mike and Lazarevich, Ivan and Sinnadurai, Nish and Lie, Sean and Ioannou, Yani and Thangarasa, Vithursan}, + journal={arXiv preprint arXiv:2510.13999}, + year={2025} +} +``` \ No newline at end of file diff --git a/chat_template.jinja b/chat_template.jinja new file mode 100755 index 0000000000000000000000000000000000000000..41478957aca7a04b7321022e7d1f73de5badd995 --- /dev/null +++ b/chat_template.jinja @@ -0,0 +1,103 @@ +[gMASK] +{%- if tools -%} +<|system|> +# Tools + +You may call one or more functions to assist with the user query. + +You are provided with function signatures within XML tags: + +{% for tool in tools %} +{{ tool | tojson(ensure_ascii=False) }} +{% endfor %} + + +For each function call, output the function name and arguments within the following XML format: +{function-name} +{arg-key-1} +{arg-value-1} +{arg-key-2} +{arg-value-2} +... +{%- endif -%} +{%- macro visible_text(content) -%} + {%- if content is string -%} + {{- content }} + {%- elif content is iterable and content is not mapping -%} + {%- for item in content -%} + {%- if item is mapping and item.type == 'text' -%} + {{- item.text }} + {%- elif item is string -%} + {{- item }} + {%- endif -%} + {%- endfor -%} + {%- else -%} + {{- content }} + {%- endif -%} +{%- endmacro -%} +{%- set ns = namespace(last_user_index=-1) %} +{%- for m in messages %} + {%- if m.role == 'user' %} + {% set ns.last_user_index = loop.index0 -%} + {%- endif %} +{%- endfor %} +{% for m in messages %} +{%- if m.role == 'user' -%}<|user|> +{{ visible_text(m.content) }} +{{- '/nothink' if (enable_thinking is defined and not enable_thinking and not visible_text(m.content).endswith("/nothink")) else '' -}} +{%- elif m.role == 'assistant' -%} +<|assistant|> +{%- set reasoning_content = '' %} +{%- set content = visible_text(m.content) %} +{%- if m.reasoning_content is string %} + {%- set reasoning_content = m.reasoning_content %} +{%- else %} + {%- if '' in content %} + {%- set reasoning_content = content.split('')[0].rstrip('\n').split('')[-1].lstrip('\n') %} + {%- set content = content.split('')[-1].lstrip('\n') %} + {%- endif %} +{%- endif %} +{%- if loop.index0 > ns.last_user_index and reasoning_content -%} +{{ '\n' + reasoning_content.strip() + ''}} +{%- else -%} +{{ '\n' }} +{%- endif -%} +{%- if content.strip() -%} +{{ '\n' + content.strip() }} +{%- endif -%} +{% if m.tool_calls %} +{% for tc in m.tool_calls %} +{%- if tc.function %} + {%- set tc = tc.function %} +{%- endif %} +{{ '\n' + tc.name }} +{% set _args = tc.arguments %} +{% for k, v in _args.items() %} +{{ k }} +{{ v | tojson(ensure_ascii=False) if v is not string else v }} +{% endfor %} +{% endfor %} +{% endif %} +{%- elif m.role == 'tool' -%} +{%- if m.content is string -%} +{%- if loop.first or (messages[loop.index0 - 1].role != "tool") %} + {{- '<|observation|>' }} +{%- endif %} +{{- '\n\n' }} +{{- m.content }} +{{- '\n' }} +{%- else -%} +<|observation|>{% for tr in m.content %} + + +{{ tr.output if tr.output is defined else tr }} +{% endfor -%} +{% endif -%} +{%- elif m.role == 'system' -%} +<|system|> +{{ visible_text(m.content) }} +{%- endif -%} +{%- endfor -%} +{%- if add_generation_prompt -%} + <|assistant|>{{- '\n' if (enable_thinking is defined and not enable_thinking) else '' -}} +{%- endif -%} \ No newline at end of file diff --git a/config.json b/config.json new file mode 100755 index 0000000000000000000000000000000000000000..cdaa1415675d907e1d9d114447ce18aa6a774d89 --- /dev/null +++ b/config.json @@ -0,0 +1,924 @@ +{ + "architectures": [ + "Glm4MoeForCausalLM" + ], + "attention_bias": true, + "attention_dropout": 0.0, + "eos_token_id": [ + 151329, + 151336, + 151338 + ], + "first_k_dense_replace": 3, + "head_dim": 128, + "hidden_act": "silu", + "hidden_size": 5120, + "initializer_range": 0.02, + "intermediate_size": 12288, + "max_position_embeddings": 202752, + "model_type": "glm4_moe", + "moe_intermediate_size": 1536, + "n_group": 1, + "n_routed_experts": 96, + "n_shared_experts": 1, + "norm_topk_prob": true, + "num_attention_heads": 96, + "num_experts_per_tok": 8, + "num_hidden_layers": 92, + "num_key_value_heads": 8, + "num_nextn_predict_layers": 0, + "pad_token_id": 151329, + "partial_rotary_factor": 0.5, + "quantization_config": { + "config_groups": { + "group_0": { + "input_activations": { + "actorder": null, + "block_structure": null, + "dynamic": true, + "group_size": null, + "num_bits": 8, + "observer": null, + "observer_kwargs": {}, + "strategy": "token", + "symmetric": true, + "type": "float" + }, + "output_activations": null, + "targets": [ + "Linear" + ], + "weights": { + "actorder": null, + "block_structure": null, + "dynamic": false, + "group_size": null, + "num_bits": 8, + "observer": "minmax", + "observer_kwargs": {}, + "strategy": "channel", + "symmetric": true, + "type": "float" + } + } + }, + "format": "float-quantized", + "global_compression_ratio": null, + "ignore": [ + "model.layers.12.input_layernorm", + "model.layers.48.input_layernorm", + "model.layers.74.input_layernorm", + "model.layers.5.self_attn.q_norm", + "model.layers.11.self_attn.k_proj.bias", + "model.layers.3.self_attn.k_proj.bias", + "model.layers.53.self_attn.q_norm", + "model.layers.52.mlp.gate", + "model.layers.29.mlp.gate.e_score_correction_bias", + "model.layers.34.post_attention_layernorm", + "model.layers.88.post_attention_layernorm", + "model.layers.17.input_layernorm", + "model.layers.37.self_attn.k_norm", + "model.layers.9.mlp.gate.e_score_correction_bias", + "model.layers.17.self_attn.v_proj.bias", + "model.layers.82.self_attn.q_norm", + "model.layers.41.mlp.gate", + "model.layers.73.input_layernorm", + "model.layers.72.input_layernorm", + "model.layers.61.post_attention_layernorm", + "model.layers.30.input_layernorm", + "model.layers.80.mlp.gate", + "model.layers.14.input_layernorm", + "model.layers.45.self_attn.k_norm", + "model.layers.55.mlp.gate", + "model.layers.9.input_layernorm", + "model.layers.13.input_layernorm", + "model.layers.47.self_attn.q_proj.bias", + "model.layers.84.post_attention_layernorm", + "model.layers.85.self_attn.q_norm", + "model.layers.53.self_attn.v_proj.bias", + "model.layers.83.self_attn.v_proj.bias", + "model.layers.35.self_attn.v_proj.bias", + "model.layers.76.mlp.gate.e_score_correction_bias", + "model.layers.36.input_layernorm", + "model.layers.13.post_attention_layernorm", + "model.layers.71.self_attn.k_proj.bias", + "model.layers.84.mlp.gate.e_score_correction_bias", + "model.layers.46.self_attn.k_norm", + "model.layers.50.self_attn.q_norm", + "model.layers.59.self_attn.q_proj.bias", + "model.layers.5.input_layernorm", + "model.layers.29.mlp.gate", + "model.layers.73.self_attn.q_norm", + "model.layers.27.mlp.gate", + "model.layers.49.post_attention_layernorm", + "model.layers.63.self_attn.q_proj.bias", + "model.layers.7.self_attn.q_proj.bias", + "model.layers.5.self_attn.k_proj.bias", + "model.layers.70.self_attn.q_proj.bias", + "model.layers.70.self_attn.k_norm", + "model.layers.15.self_attn.q_proj.bias", + "model.layers.59.self_attn.q_norm", + "model.layers.89.self_attn.k_proj.bias", + "model.layers.89.self_attn.q_norm", + "model.layers.42.mlp.gate.e_score_correction_bias", + "model.layers.62.self_attn.q_norm", + "model.layers.18.input_layernorm", + "model.layers.79.post_attention_layernorm", + "model.layers.63.self_attn.k_norm", + "model.layers.58.input_layernorm", + "model.layers.57.mlp.gate", + "model.layers.32.self_attn.q_norm", + "model.layers.34.self_attn.v_proj.bias", + "model.layers.83.self_attn.k_proj.bias", + "model.layers.7.input_layernorm", + "model.layers.72.self_attn.k_norm", + "model.layers.55.self_attn.v_proj.bias", + "model.layers.2.self_attn.k_proj.bias", + "model.layers.23.post_attention_layernorm", + "model.layers.60.post_attention_layernorm", + "model.layers.33.self_attn.k_proj.bias", + "model.layers.85.self_attn.k_proj.bias", + "model.layers.83.mlp.gate.e_score_correction_bias", + "model.layers.20.self_attn.k_proj.bias", + "model.layers.11.mlp.gate", + "model.layers.82.input_layernorm", + "model.layers.83.self_attn.q_norm", + "model.layers.9.post_attention_layernorm", + "model.layers.55.mlp.gate.e_score_correction_bias", + "model.layers.21.mlp.gate", + "model.layers.41.mlp.gate.e_score_correction_bias", + "model.layers.10.mlp.gate.e_score_correction_bias", + "model.layers.49.mlp.gate", + "model.layers.56.self_attn.k_norm", + "model.layers.86.self_attn.q_norm", + "model.layers.89.input_layernorm", + "model.layers.16.input_layernorm", + "model.layers.25.mlp.gate.e_score_correction_bias", + "model.layers.8.self_attn.k_norm", + "model.layers.72.post_attention_layernorm", + "model.layers.92.self_attn.q_norm", + "model.layers.73.post_attention_layernorm", + "model.layers.71.mlp.gate.e_score_correction_bias", + "model.layers.42.input_layernorm", + "model.layers.25.self_attn.k_norm", + "model.layers.50.self_attn.q_proj.bias", + "model.layers.13.self_attn.k_norm", + "model.layers.65.mlp.gate.e_score_correction_bias", + "model.layers.9.self_attn.k_norm", + "model.layers.3.input_layernorm", + "model.layers.90.self_attn.q_proj.bias", + "model.layers.35.self_attn.q_norm", + "model.layers.74.mlp.gate", + "model.layers.21.self_attn.k_norm", + "model.layers.92.mlp.gate", + "model.layers.15.mlp.gate.e_score_correction_bias", + "model.layers.61.self_attn.k_proj.bias", + "model.layers.63.self_attn.q_norm", + "model.layers.73.self_attn.q_proj.bias", + "model.layers.18.self_attn.q_norm", + "model.layers.45.mlp.gate", + "model.layers.52.self_attn.v_proj.bias", + "model.layers.22.post_attention_layernorm", + "model.layers.85.self_attn.q_proj.bias", + "model.layers.3.self_attn.k_norm", + "model.layers.54.self_attn.v_proj.bias", + "model.layers.79.self_attn.v_proj.bias", + "model.layers.31.self_attn.q_proj.bias", + "model.layers.2.input_layernorm", + "model.layers.57.self_attn.q_norm", + "model.layers.65.self_attn.k_norm", + "model.layers.60.input_layernorm", + "model.layers.70.self_attn.v_proj.bias", + "model.layers.87.post_attention_layernorm", + "model.layers.83.post_attention_layernorm", + "model.layers.51.self_attn.k_proj.bias", + "model.layers.23.self_attn.k_norm", + "model.layers.49.self_attn.k_norm", + "model.layers.76.self_attn.k_proj.bias", + "model.layers.7.self_attn.q_norm", + "model.layers.54.mlp.gate", + "model.layers.63.post_attention_layernorm", + "model.layers.27.self_attn.k_proj.bias", + "model.layers.81.input_layernorm", + "model.layers.66.post_attention_layernorm", + "model.layers.84.self_attn.q_proj.bias", + "model.layers.36.self_attn.k_proj.bias", + "model.layers.30.self_attn.v_proj.bias", + "model.layers.48.self_attn.k_norm", + "model.layers.62.mlp.gate", + "model.layers.8.mlp.gate", + "model.layers.11.mlp.gate.e_score_correction_bias", + "model.layers.28.mlp.gate", + "model.layers.30.mlp.gate.e_score_correction_bias", + "model.layers.43.input_layernorm", + "model.layers.3.self_attn.q_proj.bias", + "model.layers.88.input_layernorm", + "model.layers.5.post_attention_layernorm", + "model.layers.35.self_attn.k_norm", + "model.layers.56.post_attention_layernorm", + "model.layers.30.mlp.gate", + "model.layers.67.self_attn.q_proj.bias", + "model.layers.58.post_attention_layernorm", + "model.layers.80.post_attention_layernorm", + "model.layers.88.self_attn.v_proj.bias", + "model.layers.90.self_attn.v_proj.bias", + "model.layers.2.self_attn.k_norm", + "model.layers.67.mlp.gate.e_score_correction_bias", + "model.layers.31.input_layernorm", + "model.layers.26.post_attention_layernorm", + "model.layers.81.self_attn.k_proj.bias", + "model.layers.51.self_attn.q_norm", + "model.layers.7.self_attn.k_proj.bias", + "model.layers.76.self_attn.q_proj.bias", + "model.layers.1.self_attn.k_norm", + "model.layers.34.mlp.gate.e_score_correction_bias", + "model.layers.80.self_attn.k_proj.bias", + "model.layers.0.self_attn.k_norm", + "model.layers.44.self_attn.q_norm", + "model.layers.29.input_layernorm", + "model.layers.77.self_attn.v_proj.bias", + "model.layers.73.mlp.gate", + "model.layers.31.self_attn.v_proj.bias", + "model.layers.46.self_attn.q_proj.bias", + "model.layers.62.self_attn.k_norm", + "model.layers.87.self_attn.k_proj.bias", + "model.layers.58.self_attn.k_norm", + "model.layers.66.mlp.gate.e_score_correction_bias", + "model.layers.32.mlp.gate.e_score_correction_bias", + "model.layers.73.self_attn.k_proj.bias", + "model.layers.85.mlp.gate.e_score_correction_bias", + "model.layers.23.mlp.gate", + "model.layers.72.self_attn.q_proj.bias", + "model.layers.25.mlp.gate", + "model.layers.43.mlp.gate.e_score_correction_bias", + "model.layers.50.mlp.gate.e_score_correction_bias", + "model.layers.6.mlp.gate.e_score_correction_bias", + "model.layers.14.mlp.gate", + "model.layers.50.self_attn.v_proj.bias", + "model.layers.33.self_attn.k_norm", + "model.layers.92.self_attn.q_proj.bias", + "model.layers.6.self_attn.k_proj.bias", + "model.layers.43.mlp.gate", + "model.layers.68.self_attn.q_proj.bias", + "model.layers.33.self_attn.v_proj.bias", + "model.layers.41.self_attn.k_norm", + "model.layers.32.self_attn.k_norm", + "model.layers.28.post_attention_layernorm", + "model.layers.61.self_attn.v_proj.bias", + "model.layers.16.post_attention_layernorm", + "model.layers.48.mlp.gate.e_score_correction_bias", + "model.layers.4.self_attn.v_proj.bias", + "model.layers.88.self_attn.k_norm", + "model.layers.68.self_attn.k_norm", + "model.layers.77.self_attn.q_proj.bias", + "model.layers.8.self_attn.v_proj.bias", + "model.layers.20.mlp.gate.e_score_correction_bias", + "model.layers.86.self_attn.k_norm", + "model.layers.36.self_attn.v_proj.bias", + "model.layers.71.input_layernorm", + "model.layers.72.mlp.gate", + "model.layers.24.self_attn.k_norm", + "model.layers.20.self_attn.q_norm", + "model.layers.62.post_attention_layernorm", + "model.layers.3.self_attn.v_proj.bias", + "model.layers.25.input_layernorm", + "model.layers.20.self_attn.v_proj.bias", + "model.layers.64.self_attn.v_proj.bias", + "model.layers.19.self_attn.k_proj.bias", + "model.layers.63.mlp.gate.e_score_correction_bias", + "model.layers.92.embed_tokens", + "model.layers.62.self_attn.q_proj.bias", + "model.layers.69.self_attn.q_norm", + "model.layers.9.mlp.gate", + "model.layers.62.mlp.gate.e_score_correction_bias", + "model.layers.35.self_attn.k_proj.bias", + "model.layers.1.self_attn.q_proj.bias", + "model.layers.40.self_attn.q_norm", + "model.layers.26.input_layernorm", + "model.layers.50.self_attn.k_proj.bias", + "model.layers.39.input_layernorm", + "model.layers.28.self_attn.k_proj.bias", + "model.layers.39.self_attn.q_proj.bias", + "model.layers.5.mlp.gate", + "model.layers.56.input_layernorm", + "model.layers.60.self_attn.k_norm", + "model.layers.76.self_attn.q_norm", + "lm_head", + "model.layers.69.self_attn.k_norm", + "model.layers.23.self_attn.q_proj.bias", + "model.layers.58.self_attn.q_proj.bias", + "model.norm", + "model.layers.53.self_attn.k_proj.bias", + "model.layers.73.self_attn.v_proj.bias", + "model.layers.33.self_attn.q_proj.bias", + "model.layers.64.input_layernorm", + "model.layers.15.mlp.gate", + "model.layers.55.self_attn.q_proj.bias", + "model.layers.55.self_attn.k_norm", + "model.layers.27.mlp.gate.e_score_correction_bias", + "model.layers.68.input_layernorm", + "model.layers.30.self_attn.q_norm", + "model.layers.82.post_attention_layernorm", + "model.layers.29.self_attn.q_proj.bias", + "model.layers.79.self_attn.k_proj.bias", + "model.layers.16.self_attn.k_proj.bias", + "model.layers.71.self_attn.k_norm", + "model.layers.16.self_attn.k_norm", + "model.layers.75.self_attn.k_proj.bias", + "model.layers.52.self_attn.q_norm", + "model.layers.87.self_attn.v_proj.bias", + "model.layers.23.self_attn.k_proj.bias", + "model.layers.24.post_attention_layernorm", + "model.layers.40.self_attn.q_proj.bias", + "model.layers.46.mlp.gate.e_score_correction_bias", + "model.layers.54.input_layernorm", + "model.layers.75.post_attention_layernorm", + "model.layers.72.self_attn.k_proj.bias", + "model.layers.82.self_attn.q_proj.bias", + "model.layers.75.mlp.gate.e_score_correction_bias", + "model.layers.27.self_attn.q_norm", + "model.layers.39.self_attn.q_norm", + "model.layers.45.self_attn.k_proj.bias", + "model.layers.90.self_attn.k_proj.bias", + "model.layers.65.self_attn.q_proj.bias", + "model.layers.5.self_attn.k_norm", + "model.layers.89.self_attn.q_proj.bias", + "model.layers.10.self_attn.k_proj.bias", + "model.layers.86.self_attn.v_proj.bias", + "model.layers.89.mlp.gate.e_score_correction_bias", + "model.layers.42.self_attn.k_norm", + "model.layers.57.self_attn.k_proj.bias", + "model.layers.68.post_attention_layernorm", + "model.layers.1.self_attn.v_proj.bias", + "model.layers.38.self_attn.k_proj.bias", + "model.layers.75.self_attn.v_proj.bias", + "model.layers.44.self_attn.q_proj.bias", + "model.layers.8.mlp.gate.e_score_correction_bias", + "model.layers.55.input_layernorm", + "model.layers.7.self_attn.v_proj.bias", + "model.layers.51.self_attn.k_norm", + "model.layers.19.mlp.gate", + "model.layers.12.self_attn.q_norm", + "model.layers.78.self_attn.k_norm", + "model.layers.61.self_attn.q_norm", + "model.layers.18.mlp.gate", + "model.layers.8.self_attn.q_norm", + "model.layers.7.self_attn.k_norm", + "model.layers.84.self_attn.q_norm", + "model.layers.44.self_attn.k_proj.bias", + "model.layers.38.self_attn.q_proj.bias", + "model.layers.37.mlp.gate.e_score_correction_bias", + "model.layers.34.mlp.gate", + "model.layers.92.eh_proj", + "model.layers.75.self_attn.q_proj.bias", + "model.layers.61.mlp.gate.e_score_correction_bias", + "model.layers.33.mlp.gate.e_score_correction_bias", + "model.layers.81.mlp.gate.e_score_correction_bias", + "model.layers.47.self_attn.k_proj.bias", + "model.layers.72.self_attn.q_norm", + "model.layers.62.self_attn.k_proj.bias", + "model.layers.33.self_attn.q_norm", + "model.layers.24.mlp.gate", + "model.layers.51.input_layernorm", + "model.layers.42.self_attn.k_proj.bias", + "model.layers.43.self_attn.k_norm", + "model.layers.46.self_attn.k_proj.bias", + "model.layers.66.input_layernorm", + "model.layers.31.mlp.gate.e_score_correction_bias", + "model.layers.77.post_attention_layernorm", + "model.layers.52.self_attn.k_norm", + "model.layers.4.mlp.gate", + "model.layers.51.post_attention_layernorm", + "model.layers.15.self_attn.k_norm", + "model.layers.60.self_attn.k_proj.bias", + "model.layers.6.self_attn.q_norm", + "model.layers.85.mlp.gate", + "model.layers.66.self_attn.k_norm", + "model.layers.22.self_attn.v_proj.bias", + "model.layers.36.self_attn.k_norm", + "model.layers.8.input_layernorm", + "model.layers.19.mlp.gate.e_score_correction_bias", + "model.layers.25.self_attn.k_proj.bias", + "model.layers.31.self_attn.q_norm", + "model.layers.16.mlp.gate.e_score_correction_bias", + "model.layers.57.self_attn.v_proj.bias", + "model.layers.35.mlp.gate", + "model.layers.75.input_layernorm", + "model.layers.2.self_attn.v_proj.bias", + "model.layers.9.self_attn.v_proj.bias", + "model.layers.80.self_attn.q_proj.bias", + "model.layers.19.self_attn.v_proj.bias", + "model.layers.4.post_attention_layernorm", + "model.layers.65.input_layernorm", + "model.layers.82.mlp.gate.e_score_correction_bias", + "model.layers.69.post_attention_layernorm", + "model.layers.21.self_attn.q_norm", + "model.layers.18.post_attention_layernorm", + "model.layers.81.self_attn.q_proj.bias", + "model.layers.77.input_layernorm", + "model.layers.17.self_attn.q_norm", + "model.layers.80.input_layernorm", + "model.layers.13.mlp.gate.e_score_correction_bias", + "model.layers.0.post_attention_layernorm", + "model.layers.19.post_attention_layernorm", + "model.layers.91.self_attn.v_proj.bias", + "model.layers.77.mlp.gate", + "model.layers.11.self_attn.q_norm", + "model.layers.92.hnorm", + "model.layers.11.self_attn.q_proj.bias", + "model.layers.47.self_attn.q_norm", + "model.layers.42.self_attn.v_proj.bias", + "model.layers.90.input_layernorm", + "model.layers.78.input_layernorm", + "model.layers.11.self_attn.v_proj.bias", + "model.layers.80.self_attn.q_norm", + "model.layers.83.input_layernorm", + "model.layers.43.self_attn.q_norm", + "model.layers.91.post_attention_layernorm", + "model.layers.50.mlp.gate", + "model.layers.48.self_attn.k_proj.bias", + "model.layers.70.mlp.gate.e_score_correction_bias", + "model.layers.14.self_attn.q_proj.bias", + "model.layers.31.self_attn.k_proj.bias", + "model.layers.47.self_attn.v_proj.bias", + "model.layers.12.self_attn.k_proj.bias", + "model.layers.12.mlp.gate", + "model.layers.34.self_attn.q_norm", + "model.layers.62.self_attn.v_proj.bias", + "model.layers.26.mlp.gate.e_score_correction_bias", + "model.layers.45.mlp.gate.e_score_correction_bias", + "model.layers.77.self_attn.q_norm", + "model.layers.47.self_attn.k_norm", + "model.layers.53.post_attention_layernorm", + "model.layers.7.post_attention_layernorm", + "model.layers.90.post_attention_layernorm", + "model.layers.40.mlp.gate.e_score_correction_bias", + "model.layers.63.input_layernorm", + "model.layers.52.mlp.gate.e_score_correction_bias", + "model.layers.92.input_layernorm", + "model.layers.44.self_attn.v_proj.bias", + "model.layers.46.mlp.gate", + "model.layers.18.self_attn.k_proj.bias", + "model.layers.62.input_layernorm", + "model.layers.24.self_attn.v_proj.bias", + "model.layers.40.post_attention_layernorm", + "model.layers.8.post_attention_layernorm", + "model.layers.82.mlp.gate", + "model.layers.17.post_attention_layernorm", + "model.layers.45.self_attn.v_proj.bias", + "model.layers.84.self_attn.k_proj.bias", + "model.layers.15.self_attn.k_proj.bias", + "model.layers.0.self_attn.q_proj.bias", + "model.layers.92.shared_head.norm", + "model.layers.51.self_attn.q_proj.bias", + "model.layers.22.mlp.gate", + "model.layers.4.self_attn.k_norm", + "model.layers.84.input_layernorm", + "model.layers.30.self_attn.k_proj.bias", + "model.layers.12.self_attn.v_proj.bias", + "model.layers.37.input_layernorm", + "model.layers.37.self_attn.q_norm", + "model.layers.45.self_attn.q_norm", + "model.layers.72.self_attn.v_proj.bias", + "model.layers.64.self_attn.k_norm", + "model.layers.43.self_attn.k_proj.bias", + "model.layers.38.mlp.gate.e_score_correction_bias", + "model.layers.79.input_layernorm", + "model.layers.48.mlp.gate", + "model.layers.52.self_attn.q_proj.bias", + "model.layers.46.self_attn.v_proj.bias", + "model.layers.81.mlp.gate", + "model.layers.71.self_attn.q_proj.bias", + "model.layers.50.self_attn.k_norm", + "model.layers.18.self_attn.v_proj.bias", + "model.layers.53.input_layernorm", + "model.layers.64.self_attn.q_proj.bias", + "model.layers.12.self_attn.k_norm", + "model.embed_tokens", + "model.layers.78.self_attn.k_proj.bias", + "model.layers.40.self_attn.k_norm", + "model.layers.11.input_layernorm", + "model.layers.22.input_layernorm", + "model.layers.39.mlp.gate", + "model.layers.30.self_attn.k_norm", + "model.layers.71.mlp.gate", + "model.layers.71.post_attention_layernorm", + "model.layers.78.self_attn.q_norm", + "model.layers.87.mlp.gate", + "model.layers.14.self_attn.q_norm", + "model.layers.42.post_attention_layernorm", + "model.layers.3.post_attention_layernorm", + "model.layers.92.self_attn.k_proj.bias", + "model.layers.3.mlp.gate", + "model.layers.89.mlp.gate", + "model.layers.86.self_attn.k_proj.bias", + "model.layers.87.self_attn.q_proj.bias", + "model.layers.63.self_attn.k_proj.bias", + "model.layers.87.input_layernorm", + "model.layers.10.self_attn.q_proj.bias", + "model.layers.81.post_attention_layernorm", + "model.layers.39.self_attn.v_proj.bias", + "model.layers.43.post_attention_layernorm", + "model.layers.76.mlp.gate", + "model.layers.28.self_attn.v_proj.bias", + "model.layers.57.self_attn.q_proj.bias", + "model.layers.21.mlp.gate.e_score_correction_bias", + "model.layers.75.self_attn.q_norm", + "model.layers.58.self_attn.k_proj.bias", + "model.layers.52.input_layernorm", + "model.layers.0.self_attn.k_proj.bias", + "model.layers.78.mlp.gate.e_score_correction_bias", + "model.layers.85.self_attn.k_norm", + "model.layers.42.self_attn.q_norm", + "model.layers.25.self_attn.v_proj.bias", + "model.layers.36.post_attention_layernorm", + "model.layers.4.self_attn.q_norm", + "model.layers.16.self_attn.q_norm", + "model.layers.41.self_attn.k_proj.bias", + "model.layers.5.self_attn.q_proj.bias", + "model.layers.28.self_attn.q_norm", + "model.layers.41.self_attn.q_proj.bias", + "model.layers.68.mlp.gate.e_score_correction_bias", + "model.layers.26.self_attn.k_proj.bias", + "model.layers.86.input_layernorm", + "model.layers.35.mlp.gate.e_score_correction_bias", + "model.layers.69.input_layernorm", + "model.layers.77.mlp.gate.e_score_correction_bias", + "model.layers.10.self_attn.q_norm", + "model.layers.32.post_attention_layernorm", + "model.layers.54.self_attn.q_norm", + "model.layers.45.post_attention_layernorm", + "model.layers.45.input_layernorm", + "model.layers.15.post_attention_layernorm", + "model.layers.64.post_attention_layernorm", + "model.layers.15.self_attn.v_proj.bias", + "model.layers.24.self_attn.k_proj.bias", + "model.layers.64.self_attn.k_proj.bias", + "model.layers.14.post_attention_layernorm", + "model.layers.40.mlp.gate", + "model.layers.50.input_layernorm", + "model.layers.21.self_attn.v_proj.bias", + "model.layers.66.self_attn.q_proj.bias", + "model.layers.78.mlp.gate", + "model.layers.2.self_attn.q_proj.bias", + "model.layers.12.self_attn.q_proj.bias", + "model.layers.7.mlp.gate", + "model.layers.35.post_attention_layernorm", + "model.layers.52.post_attention_layernorm", + "model.layers.30.post_attention_layernorm", + "model.layers.59.input_layernorm", + "model.layers.86.post_attention_layernorm", + "model.layers.0.self_attn.v_proj.bias", + "model.layers.24.mlp.gate.e_score_correction_bias", + "model.layers.34.self_attn.k_norm", + "model.layers.4.self_attn.k_proj.bias", + "model.layers.74.self_attn.q_proj.bias", + "model.layers.78.self_attn.v_proj.bias", + "model.layers.49.self_attn.q_proj.bias", + "model.layers.53.mlp.gate.e_score_correction_bias", + "model.layers.60.self_attn.v_proj.bias", + "model.layers.59.mlp.gate", + "model.layers.61.mlp.gate", + "model.layers.26.self_attn.q_norm", + "model.layers.1.input_layernorm", + "model.layers.33.mlp.gate", + "model.layers.91.mlp.gate", + "model.layers.26.self_attn.q_proj.bias", + "model.layers.83.mlp.gate", + "model.layers.12.mlp.gate.e_score_correction_bias", + "model.layers.36.self_attn.q_norm", + "model.layers.91.mlp.gate.e_score_correction_bias", + "model.layers.25.self_attn.q_norm", + "model.layers.91.self_attn.k_norm", + "model.layers.20.self_attn.k_norm", + "model.layers.8.self_attn.k_proj.bias", + "model.layers.29.self_attn.v_proj.bias", + "model.layers.82.self_attn.k_proj.bias", + "model.layers.89.post_attention_layernorm", + "model.layers.29.self_attn.k_norm", + "model.layers.88.self_attn.q_norm", + "model.layers.48.self_attn.q_proj.bias", + "model.layers.22.self_attn.k_norm", + "model.layers.27.self_attn.q_proj.bias", + "model.layers.23.mlp.gate.e_score_correction_bias", + "model.layers.1.post_attention_layernorm", + "model.layers.29.post_attention_layernorm", + "model.layers.58.mlp.gate", + "model.layers.27.input_layernorm", + "model.layers.66.self_attn.k_proj.bias", + "model.layers.87.mlp.gate.e_score_correction_bias", + "model.layers.22.self_attn.q_norm", + "model.layers.4.mlp.gate.e_score_correction_bias", + "model.layers.51.mlp.gate", + "model.layers.90.mlp.gate.e_score_correction_bias", + "model.layers.3.self_attn.q_norm", + "model.layers.33.input_layernorm", + "model.layers.91.self_attn.k_proj.bias", + "model.layers.8.self_attn.q_proj.bias", + "model.layers.12.post_attention_layernorm", + "model.layers.3.mlp.gate.e_score_correction_bias", + "model.layers.69.mlp.gate.e_score_correction_bias", + "model.layers.17.self_attn.k_proj.bias", + "model.layers.82.self_attn.k_norm", + "model.layers.10.post_attention_layernorm", + "model.layers.84.mlp.gate", + "model.layers.6.input_layernorm", + "model.layers.70.mlp.gate", + "model.layers.33.post_attention_layernorm", + "model.layers.31.self_attn.k_norm", + "model.layers.88.self_attn.q_proj.bias", + "model.layers.80.self_attn.k_norm", + "model.layers.36.mlp.gate.e_score_correction_bias", + "model.layers.56.self_attn.v_proj.bias", + "model.layers.1.self_attn.k_proj.bias", + "model.layers.44.self_attn.k_norm", + "model.layers.60.mlp.gate.e_score_correction_bias", + "model.layers.32.self_attn.k_proj.bias", + "model.layers.69.self_attn.k_proj.bias", + "model.layers.4.input_layernorm", + "model.layers.15.self_attn.q_norm", + "model.layers.79.self_attn.k_norm", + "model.layers.4.self_attn.q_proj.bias", + "model.layers.51.mlp.gate.e_score_correction_bias", + "model.layers.57.self_attn.k_norm", + "model.layers.90.self_attn.k_norm", + "model.layers.27.self_attn.k_norm", + "model.layers.50.post_attention_layernorm", + "model.layers.21.self_attn.q_proj.bias", + "model.layers.10.input_layernorm", + "model.layers.64.self_attn.q_norm", + "model.layers.67.self_attn.k_proj.bias", + "model.layers.85.self_attn.v_proj.bias", + "model.layers.14.mlp.gate.e_score_correction_bias", + "model.layers.32.mlp.gate", + "model.layers.67.self_attn.v_proj.bias", + "model.layers.26.self_attn.k_norm", + "model.layers.19.self_attn.q_norm", + "model.layers.22.mlp.gate.e_score_correction_bias", + "model.layers.68.self_attn.q_norm", + "model.layers.86.mlp.gate.e_score_correction_bias", + "model.layers.20.post_attention_layernorm", + "model.layers.21.post_attention_layernorm", + "model.layers.49.self_attn.v_proj.bias", + "model.layers.38.mlp.gate", + "model.layers.82.self_attn.v_proj.bias", + "model.layers.42.mlp.gate", + "model.layers.21.input_layernorm", + "model.layers.22.self_attn.k_proj.bias", + "model.layers.54.self_attn.q_proj.bias", + "model.layers.76.post_attention_layernorm", + "model.layers.67.input_layernorm", + "model.layers.65.self_attn.v_proj.bias", + "model.layers.67.self_attn.q_norm", + "model.layers.79.self_attn.q_norm", + "model.layers.47.input_layernorm", + "model.layers.79.mlp.gate.e_score_correction_bias", + "model.layers.76.self_attn.k_norm", + "model.layers.60.mlp.gate", + "model.layers.20.self_attn.q_proj.bias", + "model.layers.11.post_attention_layernorm", + "model.layers.24.self_attn.q_norm", + "model.layers.53.self_attn.k_norm", + "model.layers.53.self_attn.q_proj.bias", + "model.layers.75.self_attn.k_norm", + "model.layers.28.self_attn.q_proj.bias", + "model.layers.65.self_attn.q_norm", + "model.layers.39.self_attn.k_proj.bias", + "model.layers.85.input_layernorm", + "model.layers.17.self_attn.q_proj.bias", + "model.layers.65.post_attention_layernorm", + "model.layers.68.mlp.gate", + "model.layers.13.self_attn.q_norm", + "model.layers.81.self_attn.v_proj.bias", + "model.layers.58.self_attn.q_norm", + "model.layers.34.input_layernorm", + "model.layers.25.self_attn.q_proj.bias", + "model.layers.77.self_attn.k_proj.bias", + "model.layers.39.mlp.gate.e_score_correction_bias", + "model.layers.0.input_layernorm", + "model.layers.76.self_attn.v_proj.bias", + "model.layers.6.self_attn.k_norm", + "model.layers.74.self_attn.k_proj.bias", + "model.layers.92.self_attn.k_norm", + "model.layers.37.self_attn.k_proj.bias", + "model.layers.18.self_attn.k_norm", + "model.layers.48.self_attn.q_norm", + "model.layers.84.self_attn.v_proj.bias", + "model.layers.47.mlp.gate.e_score_correction_bias", + "model.layers.49.self_attn.q_norm", + "model.layers.51.self_attn.v_proj.bias", + "model.layers.61.input_layernorm", + "model.layers.28.self_attn.k_norm", + "model.layers.32.self_attn.v_proj.bias", + "model.layers.55.self_attn.q_norm", + "model.layers.55.self_attn.k_proj.bias", + "model.layers.47.post_attention_layernorm", + "model.layers.69.self_attn.q_proj.bias", + "model.layers.28.input_layernorm", + "model.layers.40.self_attn.v_proj.bias", + "model.layers.19.input_layernorm", + "model.layers.83.self_attn.q_proj.bias", + "model.layers.66.self_attn.v_proj.bias", + "model.layers.59.mlp.gate.e_score_correction_bias", + "model.layers.23.self_attn.v_proj.bias", + "model.layers.0.self_attn.q_norm", + "model.layers.5.mlp.gate.e_score_correction_bias", + "model.layers.37.self_attn.v_proj.bias", + "model.layers.2.post_attention_layernorm", + "model.layers.5.self_attn.v_proj.bias", + "model.layers.34.self_attn.k_proj.bias", + "model.layers.38.self_attn.v_proj.bias", + "model.layers.81.self_attn.k_norm", + "model.layers.32.input_layernorm", + "model.layers.10.self_attn.k_norm", + "model.layers.54.self_attn.k_norm", + "model.layers.70.input_layernorm", + "model.layers.41.self_attn.v_proj.bias", + "model.layers.74.post_attention_layernorm", + "model.layers.35.input_layernorm", + "model.layers.7.mlp.gate.e_score_correction_bias", + "model.layers.6.post_attention_layernorm", + "model.layers.86.mlp.gate", + "model.layers.20.mlp.gate", + "model.layers.31.mlp.gate", + "model.layers.2.self_attn.q_norm", + "model.layers.23.self_attn.q_norm", + "model.layers.6.self_attn.q_proj.bias", + "model.layers.83.self_attn.k_norm", + "model.layers.38.input_layernorm", + "model.layers.60.self_attn.q_norm", + "model.layers.91.self_attn.q_norm", + "model.layers.44.input_layernorm", + "model.layers.67.post_attention_layernorm", + "model.layers.27.self_attn.v_proj.bias", + "model.layers.15.input_layernorm", + "model.layers.26.self_attn.v_proj.bias", + "model.layers.44.post_attention_layernorm", + "model.layers.46.self_attn.q_norm", + "model.layers.58.self_attn.v_proj.bias", + "model.layers.67.self_attn.k_norm", + "model.layers.38.self_attn.k_norm", + "model.layers.56.mlp.gate.e_score_correction_bias", + "model.layers.73.self_attn.k_norm", + "model.layers.80.mlp.gate.e_score_correction_bias", + "model.layers.37.self_attn.q_proj.bias", + "model.layers.17.mlp.gate", + "model.layers.19.self_attn.k_norm", + "model.layers.53.mlp.gate", + "model.layers.21.self_attn.k_proj.bias", + "model.layers.56.self_attn.q_norm", + "model.layers.59.self_attn.v_proj.bias", + "model.layers.64.mlp.gate.e_score_correction_bias", + "model.layers.86.self_attn.q_proj.bias", + "model.layers.46.input_layernorm", + "model.layers.6.mlp.gate", + "model.layers.40.self_attn.k_proj.bias", + "model.layers.60.self_attn.q_proj.bias", + "model.layers.6.self_attn.v_proj.bias", + "model.layers.47.mlp.gate", + "model.layers.32.self_attn.q_proj.bias", + "model.layers.17.mlp.gate.e_score_correction_bias", + "model.layers.29.self_attn.k_proj.bias", + "model.layers.44.mlp.gate.e_score_correction_bias", + "model.layers.57.mlp.gate.e_score_correction_bias", + "model.layers.87.self_attn.q_norm", + "model.layers.71.self_attn.q_norm", + "model.layers.52.self_attn.k_proj.bias", + "model.layers.88.mlp.gate.e_score_correction_bias", + "model.layers.68.self_attn.k_proj.bias", + "model.layers.16.self_attn.q_proj.bias", + "model.layers.48.self_attn.v_proj.bias", + "model.layers.74.self_attn.q_norm", + "model.layers.34.self_attn.q_proj.bias", + "model.layers.1.self_attn.q_norm", + "model.layers.63.mlp.gate", + "model.layers.92.enorm", + "model.layers.14.self_attn.v_proj.bias", + "model.layers.54.post_attention_layernorm", + "model.layers.41.self_attn.q_norm", + "model.layers.37.post_attention_layernorm", + "model.layers.61.self_attn.q_proj.bias", + "model.layers.91.input_layernorm", + "model.layers.11.self_attn.k_norm", + "model.layers.43.self_attn.q_proj.bias", + "model.layers.18.mlp.gate.e_score_correction_bias", + "model.layers.59.post_attention_layernorm", + "model.layers.9.self_attn.q_norm", + "model.layers.74.self_attn.k_norm", + "model.layers.43.self_attn.v_proj.bias", + "model.layers.49.self_attn.k_proj.bias", + "model.layers.56.self_attn.k_proj.bias", + "model.layers.14.self_attn.k_proj.bias", + "model.layers.65.mlp.gate", + "model.layers.56.self_attn.q_proj.bias", + "model.layers.24.self_attn.q_proj.bias", + "model.layers.71.self_attn.v_proj.bias", + "model.layers.54.mlp.gate.e_score_correction_bias", + "model.layers.39.post_attention_layernorm", + "model.layers.91.self_attn.q_proj.bias", + "model.layers.92.mlp.gate.e_score_correction_bias", + "model.layers.46.post_attention_layernorm", + "model.layers.92.self_attn.v_proj.bias", + "model.layers.63.self_attn.v_proj.bias", + "model.layers.25.post_attention_layernorm", + "model.layers.45.self_attn.q_proj.bias", + "model.layers.38.self_attn.q_norm", + "model.layers.68.self_attn.v_proj.bias", + "model.layers.69.mlp.gate", + "model.layers.84.self_attn.k_norm", + "model.layers.19.self_attn.q_proj.bias", + "model.layers.38.post_attention_layernorm", + "model.layers.44.mlp.gate", + "model.layers.70.post_attention_layernorm", + "model.layers.58.mlp.gate.e_score_correction_bias", + "model.layers.28.mlp.gate.e_score_correction_bias", + "model.layers.48.post_attention_layernorm", + "model.layers.36.self_attn.q_proj.bias", + "model.layers.66.self_attn.q_norm", + "model.layers.70.self_attn.k_proj.bias", + "model.layers.57.input_layernorm", + "model.layers.42.self_attn.q_proj.bias", + "model.layers.26.mlp.gate", + "model.layers.74.self_attn.v_proj.bias", + "model.layers.59.self_attn.k_proj.bias", + "model.layers.88.mlp.gate", + "model.layers.14.self_attn.k_norm", + "model.layers.88.self_attn.k_proj.bias", + "model.layers.35.self_attn.q_proj.bias", + "model.layers.65.self_attn.k_proj.bias", + "model.layers.18.self_attn.q_proj.bias", + "model.layers.30.self_attn.q_proj.bias", + "model.layers.29.self_attn.q_norm", + "model.layers.78.post_attention_layernorm", + "model.layers.79.mlp.gate", + "model.layers.27.post_attention_layernorm", + "model.layers.80.self_attn.v_proj.bias", + "model.layers.16.mlp.gate", + "model.layers.10.self_attn.v_proj.bias", + "model.layers.13.self_attn.v_proj.bias", + "model.layers.70.self_attn.q_norm", + "model.layers.73.mlp.gate.e_score_correction_bias", + "model.layers.41.post_attention_layernorm", + "model.layers.39.self_attn.k_norm", + "model.layers.10.mlp.gate", + "model.layers.36.mlp.gate", + "model.layers.92.post_attention_layernorm", + "model.layers.64.mlp.gate", + "model.layers.40.input_layernorm", + "model.layers.77.self_attn.k_norm", + "model.layers.31.post_attention_layernorm", + "model.layers.56.mlp.gate", + "model.layers.16.self_attn.v_proj.bias", + "model.layers.9.self_attn.q_proj.bias", + "model.layers.13.mlp.gate", + "model.layers.17.self_attn.k_norm", + "model.layers.22.self_attn.q_proj.bias", + "model.layers.49.mlp.gate.e_score_correction_bias", + "model.layers.9.self_attn.k_proj.bias", + "model.layers.37.mlp.gate", + "model.layers.72.mlp.gate.e_score_correction_bias", + "model.layers.90.self_attn.q_norm", + "model.layers.79.self_attn.q_proj.bias", + "model.layers.75.mlp.gate", + "model.layers.74.mlp.gate.e_score_correction_bias", + "model.layers.89.self_attn.v_proj.bias", + "model.layers.59.self_attn.k_norm", + "model.layers.76.input_layernorm", + "model.layers.66.mlp.gate", + "model.layers.55.post_attention_layernorm", + "model.layers.24.input_layernorm", + "model.layers.57.post_attention_layernorm", + "model.layers.23.input_layernorm", + "model.layers.67.mlp.gate", + "model.layers.13.self_attn.k_proj.bias", + "model.layers.69.self_attn.v_proj.bias", + "model.layers.85.post_attention_layernorm", + "model.layers.13.self_attn.q_proj.bias", + "model.layers.87.self_attn.k_norm", + "model.layers.89.self_attn.k_norm", + "model.layers.54.self_attn.k_proj.bias", + "model.layers.49.input_layernorm", + "model.layers.78.self_attn.q_proj.bias", + "model.layers.81.self_attn.q_norm", + "model.layers.41.input_layernorm", + "model.layers.90.mlp.gate", + "model.layers.61.self_attn.k_norm", + "model.layers.20.input_layernorm" + ], + "kv_cache_scheme": null, + "quant_method": "compressed-tensors", + "quantization_status": "compressed", + "sparsity_config": {} + }, + "rms_norm_eps": 1e-05, + "rope_scaling": null, + "rope_theta": 1000000, + "routed_scaling_factor": 2.5, + "tie_word_embeddings": false, + "topk_group": 1, + "torch_dtype": "bfloat16", + "transformers_version": "4.55.0", + "use_cache": true, + "use_qk_norm": true, + "vocab_size": 151552 +} diff --git a/generation_config.json b/generation_config.json new file mode 100755 index 0000000000000000000000000000000000000000..05b54ecc5cec6cc2cbd20f2ade90000d0bf0deb7 --- /dev/null +++ b/generation_config.json @@ -0,0 +1,10 @@ +{ + "_from_model_config": true, + "eos_token_id": [ + 151329, + 151336, + 151338 + ], + "pad_token_id": 151329, + "transformers_version": "4.55.0" +} diff --git a/model-00001-of-00045.safetensors b/model-00001-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..fdd7fd440b7b9313dc3631ef9bf8a6d1c5077e66 --- /dev/null +++ b/model-00001-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e9f61c6ba406a10f26e08ceb1367e01483edc05cf383d45ab9732082a5ab7c85 +size 4955138040 diff --git a/model-00002-of-00045.safetensors b/model-00002-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..1a9d799fce32374dee31ba28eb7b83b0e9b13253 --- /dev/null +++ b/model-00002-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:82ef98aade0c09877d5b0be093c3df6b005cb6ab8f05a06485b73f4380446018 +size 4999339816 diff --git a/model-00003-of-00045.safetensors b/model-00003-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..8a31c8284d2334ad2baeee00db90eb0969b517d2 --- /dev/null +++ b/model-00003-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8778cf2d1949531b671f6861e420a8c672d39bb1b96f5f9886deb920dbf5b242 +size 4996794424 diff --git a/model-00004-of-00045.safetensors b/model-00004-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..81370dca5563d801695964b09e105557ecedb64a --- /dev/null +++ b/model-00004-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ba3fab413209fcc0dabad677c98618d8ce76292eb494843aafa501a6680874a5 +size 4996794552 diff --git a/model-00005-of-00045.safetensors b/model-00005-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..6a1d057a2b0c6e66e784c3f36289b16e102242c1 --- /dev/null +++ b/model-00005-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a9cee9b1378a9e16e9eb06b03fcd50e9ce9a57e9784c051f6c766847d61f74b6 +size 4996795696 diff --git a/model-00006-of-00045.safetensors b/model-00006-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..9ee7597b2db0aa801a6d4abe96037bd42ac399dc --- /dev/null +++ b/model-00006-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8cda5a02e6220460b361f1b7486c80381ffa88a1e29b4a03281e4a611ba0b8c0 +size 4996795696 diff --git a/model-00007-of-00045.safetensors b/model-00007-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..9995c31e8256d049816bacc20dad1fa14725af91 --- /dev/null +++ b/model-00007-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b4779cc3819fc4589e13b954d1492faebc0b976668ae6e1b972c7ffcd36bb39 +size 4996795696 diff --git a/model-00008-of-00045.safetensors b/model-00008-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..d9562838ace9fd9714a798f725c98b42114c7e03 --- /dev/null +++ b/model-00008-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dc37bb4e2d662e124cc60945379b2065260614ba2b0a35872a80277fe299abdf +size 4996795696 diff --git a/model-00009-of-00045.safetensors b/model-00009-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..8c1efe4672756bc627f35cb2a2b1081da6f7a040 --- /dev/null +++ b/model-00009-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4cd4dd4b694660f064418b9e46165ad57da18c129a02d2eaeca3d6a978bbff52 +size 4996795696 diff --git a/model-00010-of-00045.safetensors b/model-00010-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..34e4f6c55a828f7b865211744741d4b7e3e853c3 --- /dev/null +++ b/model-00010-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7139a40dca4ed235683b97ae47be8b2fb147e9b8d9ddad06262aa9014fcad9ee +size 4996795728 diff --git a/model-00011-of-00045.safetensors b/model-00011-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..215bd32e3ac01807c58c2d56da8c390309148af7 --- /dev/null +++ b/model-00011-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7726392e21598cf86fbd2c9460ed20474c2f447c17099d334be5350919654246 +size 4996795768 diff --git a/model-00012-of-00045.safetensors b/model-00012-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..fd1225cbf59ce1723a4192f4856e0a7f75fbccfd --- /dev/null +++ b/model-00012-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:16716c8dccd070ab0336dc440cf821dac49a721ba52138f89e4e1debd63ef76b +size 4996795800 diff --git a/model-00013-of-00045.safetensors b/model-00013-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..30a2cf04de91f160c45b8123ae0b37097ee1bb69 --- /dev/null +++ b/model-00013-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52bd7cd342e0755d428ee2f8b74da4b06104db15fa1f2c9d5f756137982fe0b3 +size 4996795840 diff --git a/model-00014-of-00045.safetensors b/model-00014-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..b20245ecede5f965f448a1393469a431dc4c79e2 --- /dev/null +++ b/model-00014-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:709c159dbd34ab6b101534cc6dd7246cd011cb91db7d748b8e50aafa6ea2e0f1 +size 4996795872 diff --git a/model-00015-of-00045.safetensors b/model-00015-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..7308eb90b0b36b33d76a8bef568d64010b9de721 --- /dev/null +++ b/model-00015-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc619df0cdecedd28fbe259cfa981c86321ec6f63778623fc49d2b830398856b +size 4996795912 diff --git a/model-00016-of-00045.safetensors b/model-00016-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..92bec96641ef48a8f5e4f94aa2af2e647735138b --- /dev/null +++ b/model-00016-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dfc9efdf9926a1b2d479a75ca0f2529ca600018851ae6960f1c8274a665d9e8b +size 4996795944 diff --git a/model-00017-of-00045.safetensors b/model-00017-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..59d64365ccdf2b26b1c60f4156c0bf8fab0ff5f2 --- /dev/null +++ b/model-00017-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:11863df31fedb4abe13320a8dc193158e4bfb73e36a8bd7b9b5784ba6ae1654d +size 4996795984 diff --git a/model-00018-of-00045.safetensors b/model-00018-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..67cde229835fb19d21d09657c3ba94e9f507d2f0 --- /dev/null +++ b/model-00018-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9919c4e169cd9bc1432592dbd52a3d98525f4cb9618e3564bc4874898c8d5b91 +size 4997779480 diff --git a/model-00019-of-00045.safetensors b/model-00019-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..8ddb5c5003b04c311338e9e4260289ad2200467d --- /dev/null +++ b/model-00019-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:54bb330a2acb65b8b650457a343042525a08284cc5f8974bbf555ce53faac7c1 +size 4944364144 diff --git a/model-00020-of-00045.safetensors b/model-00020-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..f1b54f2d685768187d5fa984e2d00194f95e7aa7 --- /dev/null +++ b/model-00020-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b98ef021d2b38994acfbd1112a64297979b5763e4db2c37bebc385d84d63a7a0 +size 4996751624 diff --git a/model-00021-of-00045.safetensors b/model-00021-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..ee1894273dbf98053581f7535efffb0696d135ec --- /dev/null +++ b/model-00021-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:449fb921563c5b41ff6668dcc14871632d965c8c6f37d1f9e5ef85750cf13fa1 +size 4996795656 diff --git a/model-00022-of-00045.safetensors b/model-00022-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..ae615cb3d5fa00a7a499c2aab9063c8fa2ce079d --- /dev/null +++ b/model-00022-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:69442c04fbc371b5e46c4a315a7e782094df20afd6a536c43f27dcbf1a5b5258 +size 4996795688 diff --git a/model-00023-of-00045.safetensors b/model-00023-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..7efc44205f7b32af3c572f8bce9d6e23556cf115 --- /dev/null +++ b/model-00023-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5129a39864de6e747e78d593c4b55f38ee9ba0ac36cefcfb1d4a5d1f8b03cb0d +size 4996795696 diff --git a/model-00024-of-00045.safetensors b/model-00024-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..e0de5896549fa9efbe660c324cf1c659c4dcfc07 --- /dev/null +++ b/model-00024-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:920a64f25413e09b793f87136cf61130256920d568299a078d49297544374a71 +size 4996795696 diff --git a/model-00025-of-00045.safetensors b/model-00025-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..5ef2dc4e3a544edc211e5c58c8fe64191e8d2021 --- /dev/null +++ b/model-00025-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ed6f411d2b011ee87e4a4ab634c38d5b722506878b56ced8aa581d50308317df +size 4996795696 diff --git a/model-00026-of-00045.safetensors b/model-00026-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..22efa0e52763ba5b30eaa929324a7c64afff4a34 --- /dev/null +++ b/model-00026-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8d817fcdafba3c970716f8b840c8350e8b0cd46862b88fd92c9905639c02c21b +size 4996795696 diff --git a/model-00027-of-00045.safetensors b/model-00027-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..c722d4eb86c11dbf046c10f7e7fdc0ce53702efb --- /dev/null +++ b/model-00027-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c14ae69641c43e2ff453fe92b1fa05845ca184f202db7d433a739c39c7746b1d +size 4996795712 diff --git a/model-00028-of-00045.safetensors b/model-00028-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..9021466d465a94f0708ec19f37da6bae10d86289 --- /dev/null +++ b/model-00028-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:038ca9b81713414cac01b2c0fdd96d2961ce26e66c894985622604d98aec582a +size 4996795752 diff --git a/model-00029-of-00045.safetensors b/model-00029-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..92c0b5c1adb76d534add7af940fb1d0fadea6e5e --- /dev/null +++ b/model-00029-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:caf222e77a4fe366ff1d2ec04eed3cdb5f3dc6267ff69e78353826f7f12f1ac0 +size 4996795784 diff --git a/model-00030-of-00045.safetensors b/model-00030-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..678d019b8c9de557bf70d8fbc89bfbd88d638e8d --- /dev/null +++ b/model-00030-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:59a3c7d15225a9fc4d96470b4df46178fa05bf816a6c7b54eead5df74eb985f9 +size 4996795824 diff --git a/model-00031-of-00045.safetensors b/model-00031-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..194f9fa3c289a48e82cf3f28b7897e91c2035116 --- /dev/null +++ b/model-00031-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ab39cd3642d01c2eda14bbb8a9297db0a252c87efab79ef76ce94374dd6ec9a4 +size 4996795856 diff --git a/model-00032-of-00045.safetensors b/model-00032-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..9164c849726cf58290f57112030459d117b32776 --- /dev/null +++ b/model-00032-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e11cfeb6167af095af33b2c138c189d8db69794357b23bdb33a637fecc33731 +size 4996795896 diff --git a/model-00033-of-00045.safetensors b/model-00033-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..7b07297ca91018280fb8f761f81371253338c6cf --- /dev/null +++ b/model-00033-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eb62d772fb71cd7bf73727c768eedf12eada2bdff2746fbb07062896da1dd5c8 +size 4996795928 diff --git a/model-00034-of-00045.safetensors b/model-00034-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..8995a2cc92d60c2122cd777aa3242465d2ad4e16 --- /dev/null +++ b/model-00034-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:024baab11e89f2333e004cd9202875c31f3ac7307199c1199f0907b640cb9354 +size 4996795968 diff --git a/model-00035-of-00045.safetensors b/model-00035-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..164bd4942ad223457689b07d31ed8f6dd45b525e --- /dev/null +++ b/model-00035-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e557025c6fb11304b3087968df224b4df8f98d275a420252313b2213907983b1 +size 4996796000 diff --git a/model-00036-of-00045.safetensors b/model-00036-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..316c91e5fc4162d9fb7063421cd1355220b16f63 --- /dev/null +++ b/model-00036-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3bb2dcb337af37c4987955ea02ecd8690c64d25638f2cdea54e4c999ca286e8d +size 4942758568 diff --git a/model-00037-of-00045.safetensors b/model-00037-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..29546175c427b155ebc2d90f2c222605fe0c3af0 --- /dev/null +++ b/model-00037-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c1ee9454a72367952abc43fc8500807b2363a60e707fde139d5a94a3a5db84a7 +size 4999341024 diff --git a/model-00038-of-00045.safetensors b/model-00038-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..bb836a1dda8eeb1f868f3f386ba413e5d68cdec2 --- /dev/null +++ b/model-00038-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:aad4f5e0febdfd59e8b3fc6e3a8f27c9ef040bf5b523ae165c434535ab3ebb8c +size 4996795656 diff --git a/model-00039-of-00045.safetensors b/model-00039-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..69ea7f0d2a6a8e2d8672d066aae36ab04051dd20 --- /dev/null +++ b/model-00039-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:852d4bcc39597a97b089b1acae0563f307350e1a02d6be69f8c200ea75aff2f9 +size 4996795672 diff --git a/model-00040-of-00045.safetensors b/model-00040-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..2f5d86660cc8e88da1b360dc8133e819a4557d5b --- /dev/null +++ b/model-00040-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1cfe008daa0a9751345d5eb1706a5ea4077be44f899000bdaad17988f8d1ccde +size 4996795696 diff --git a/model-00041-of-00045.safetensors b/model-00041-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..4bda45780e41a30c9d1fc8dc1effbe490ac657b5 --- /dev/null +++ b/model-00041-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:720015ff58bcae4aff784b07941db6bb36cbaf6398911d09c33d9f174f2c09a9 +size 4996795696 diff --git a/model-00042-of-00045.safetensors b/model-00042-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..c2e90750279a05ee123a4768b82a65e4054ae4bd --- /dev/null +++ b/model-00042-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:469f35d4bc351506a8be9434758b1d7f31451127070c31fa06a51b0210ff17d1 +size 4996795696 diff --git a/model-00043-of-00045.safetensors b/model-00043-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..680aa81237eb4c59540ff1729429ac4d5ab7931b --- /dev/null +++ b/model-00043-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4dd87134ecd20b616dc38ba3d059b9b7b0e0f6f0b3430552899c9ad6aaed9c39 +size 4996795696 diff --git a/model-00044-of-00045.safetensors b/model-00044-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..2d0c9054f83e846c0f435ba5edeb6f26cd19d437 --- /dev/null +++ b/model-00044-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:db711e46999567ae41f127115d101f56cc2639c83c55db812c63db37823833ab +size 3860926760 diff --git a/model-00045-of-00045.safetensors b/model-00045-of-00045.safetensors new file mode 100755 index 0000000000000000000000000000000000000000..898ac7fae2f21ce370168e6a67b83112a19b0ce7 --- /dev/null +++ b/model-00045-of-00045.safetensors @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f7ed732a1f52d39814adc6a0b127bad1c816b89d7055fa07b029cba98deba7bf +size 1551892608 diff --git a/model.safetensors.index.json b/model.safetensors.index.json new file mode 100755 index 0000000000000000000000000000000000000000..476495e84882eb9bee8ffa820459e30eb160375c --- /dev/null +++ b/model.safetensors.index.json @@ -0,0 +1,53385 @@ +{ + "metadata": { + "total_parameters": 218455751680, + "total_size": 220126296768 + }, + "weight_map": { + "lm_head.weight": "model-00045-of-00045.safetensors", + "model.embed_tokens.weight": "model-00001-of-00045.safetensors", + "model.layers.0.input_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.mlp.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.k_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.k_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.o_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.q_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.q_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.0.self_attn.v_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.input_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.mlp.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.k_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.k_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.o_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.q_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.q_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.1.self_attn.v_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.10.input_layernorm.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.0.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.0.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.0.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.0.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.0.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.0.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.1.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.10.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.11.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.12.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.12.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.12.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.12.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.12.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.12.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.13.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.13.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.13.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.13.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.13.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.13.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.14.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.15.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.16.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.17.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.18.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.19.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.2.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.2.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.2.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.2.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.2.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.2.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.20.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.20.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.20.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.20.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.20.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.20.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.21.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.22.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.23.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.24.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.25.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.26.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.27.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.28.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.29.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.3.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.3.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.3.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.3.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.3.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.3.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.30.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.30.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.30.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.30.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.30.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.30.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.31.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.32.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.33.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.34.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.35.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.36.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.37.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.38.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.39.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.4.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.4.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.4.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.4.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.4.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.4.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.40.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.40.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.40.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.40.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.40.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.40.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.41.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.42.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.43.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.44.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.45.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.46.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.47.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.48.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.49.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.5.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.5.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.5.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.5.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.5.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.5.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.50.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.50.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.50.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.50.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.50.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.50.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.51.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.52.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.53.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.54.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.55.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.56.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.57.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.58.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.59.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.6.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.6.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.6.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.6.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.6.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.6.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.60.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.60.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.60.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.60.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.60.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.60.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.61.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.62.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.63.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.64.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.65.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.66.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.67.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.68.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.69.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.7.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.7.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.7.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.7.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.7.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.7.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.70.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.70.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.70.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.70.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.70.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.70.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.71.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.72.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.73.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.74.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.75.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.76.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.77.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.78.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.79.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.8.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.8.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.8.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.8.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.8.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.8.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.80.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.80.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.80.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.80.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.80.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.80.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.81.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.82.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.83.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.84.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.85.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.86.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.87.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.88.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.89.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.9.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.9.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.9.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.9.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.9.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.9.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.mlp.experts.90.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.90.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.90.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.90.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.90.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.90.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.91.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.92.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.93.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.94.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.experts.95.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.gate.e_score_correction_bias": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.gate.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.10.mlp.shared_experts.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.10.post_attention_layernorm.weight": "model-00005-of-00045.safetensors", + "model.layers.10.self_attn.k_norm.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.k_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.k_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.k_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.o_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.o_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.q_norm.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.q_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.q_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.q_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.v_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.v_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.10.self_attn.v_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.11.input_layernorm.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.0.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.1.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.10.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.11.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.12.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.13.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.14.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.15.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.16.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.17.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.18.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.19.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.2.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.20.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.21.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.22.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.23.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.24.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.25.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.26.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.27.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.28.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.29.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.3.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.30.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.31.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.32.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.33.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.34.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.35.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.36.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.37.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.38.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.39.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.4.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.40.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.41.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.42.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.43.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.44.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.45.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.46.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.47.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.48.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.49.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.5.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.50.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.51.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.52.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.53.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.54.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.55.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.56.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.57.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.58.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.59.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.6.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.60.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.61.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.62.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.63.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.64.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.65.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.66.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.67.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.68.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.69.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.7.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.70.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.71.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.72.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.73.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.74.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.75.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.76.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.77.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.78.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.79.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.8.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.80.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.81.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.82.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.83.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.84.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.85.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.86.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.87.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.88.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.89.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.9.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.90.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.91.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.92.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.93.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.94.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.experts.95.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.gate.e_score_correction_bias": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.gate.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.mlp.shared_experts.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.post_attention_layernorm.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.k_norm.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.k_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.k_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.k_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.o_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.o_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.q_norm.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.q_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.q_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.q_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.v_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.v_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.11.self_attn.v_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.input_layernorm.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.0.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.0.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.0.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.0.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.0.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.0.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.1.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.10.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.11.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.12.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.13.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.14.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.15.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.16.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.17.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.18.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.18.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.18.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.18.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.18.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.18.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.19.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.19.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.19.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.19.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.19.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.19.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.2.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.2.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.2.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.2.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.2.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.2.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.20.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.20.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.20.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.20.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.20.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.20.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.21.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.22.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.23.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.24.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.25.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.26.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.27.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.28.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.29.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.3.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.3.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.3.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.3.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.3.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.3.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.30.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.30.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.30.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.30.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.30.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.30.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.31.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.32.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.33.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.34.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.35.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.36.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.37.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.38.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.39.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.4.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.4.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.4.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.4.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.4.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.4.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.40.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.40.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.40.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.40.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.40.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.40.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.41.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.42.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.43.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.44.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.45.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.46.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.47.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.48.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.49.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.5.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.5.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.5.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.5.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.5.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.5.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.50.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.50.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.50.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.50.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.50.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.50.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.51.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.52.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.53.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.54.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.55.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.56.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.57.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.58.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.59.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.6.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.6.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.6.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.6.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.6.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.6.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.60.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.60.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.60.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.60.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.60.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.60.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.61.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.62.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.63.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.64.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.65.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.66.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.67.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.68.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.69.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.7.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.7.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.7.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.7.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.7.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.7.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.70.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.70.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.70.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.70.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.70.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.70.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.71.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.72.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.73.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.74.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.75.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.76.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.77.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.78.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.79.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.8.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.8.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.8.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.8.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.8.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.8.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.80.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.80.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.80.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.80.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.80.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.80.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.81.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.82.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.83.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.84.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.85.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.86.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.87.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.88.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.89.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.9.down_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.9.down_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.9.gate_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.9.gate_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.9.up_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.9.up_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.mlp.experts.90.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.90.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.90.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.90.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.90.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.90.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.91.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.92.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.93.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.94.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.experts.95.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.gate.e_score_correction_bias": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.gate.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.12.mlp.shared_experts.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.12.post_attention_layernorm.weight": "model-00006-of-00045.safetensors", + "model.layers.12.self_attn.k_norm.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.k_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.k_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.k_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.o_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.o_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.q_norm.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.q_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.q_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.q_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.v_proj.bias": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.v_proj.weight": "model-00005-of-00045.safetensors", + "model.layers.12.self_attn.v_proj.weight_scale": "model-00005-of-00045.safetensors", + "model.layers.13.input_layernorm.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.0.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.1.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.10.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.11.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.12.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.13.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.14.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.15.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.16.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.17.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.18.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.19.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.2.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.20.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.21.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.22.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.23.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.24.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.25.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.26.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.27.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.28.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.29.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.3.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.30.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.31.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.32.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.33.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.34.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.35.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.36.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.37.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.38.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.39.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.4.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.40.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.41.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.42.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.43.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.44.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.45.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.46.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.47.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.48.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.49.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.5.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.50.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.51.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.52.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.53.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.54.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.55.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.56.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.57.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.58.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.59.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.6.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.60.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.61.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.62.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.63.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.64.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.65.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.66.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.67.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.68.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.69.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.7.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.70.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.71.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.72.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.73.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.74.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.75.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.76.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.77.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.78.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.79.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.8.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.80.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.81.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.82.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.83.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.84.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.85.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.86.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.87.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.88.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.89.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.9.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.90.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.91.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.92.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.93.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.94.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.experts.95.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.gate.e_score_correction_bias": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.gate.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.mlp.shared_experts.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.post_attention_layernorm.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.k_norm.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.k_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.k_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.k_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.o_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.o_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.q_norm.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.q_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.q_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.q_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.v_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.v_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.13.self_attn.v_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.input_layernorm.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.0.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.0.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.0.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.0.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.0.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.0.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.1.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.10.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.11.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.12.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.13.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.14.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.15.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.16.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.17.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.18.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.19.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.2.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.20.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.21.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.22.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.23.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.24.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.24.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.24.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.24.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.24.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.24.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.25.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.25.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.25.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.25.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.25.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.25.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.26.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.27.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.28.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.29.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.3.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.3.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.3.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.3.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.3.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.3.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.30.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.30.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.30.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.30.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.30.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.30.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.31.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.32.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.33.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.34.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.35.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.36.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.37.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.38.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.39.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.4.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.4.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.4.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.4.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.4.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.4.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.40.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.40.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.40.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.40.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.40.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.40.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.41.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.42.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.43.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.44.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.45.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.46.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.47.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.48.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.49.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.5.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.5.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.5.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.5.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.5.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.5.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.50.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.50.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.50.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.50.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.50.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.50.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.51.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.52.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.53.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.54.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.55.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.56.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.57.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.58.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.59.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.6.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.6.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.6.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.6.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.6.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.6.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.60.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.60.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.60.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.60.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.60.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.60.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.61.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.62.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.63.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.64.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.65.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.66.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.67.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.68.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.69.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.7.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.7.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.7.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.7.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.7.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.7.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.70.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.70.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.70.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.70.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.70.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.70.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.71.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.72.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.73.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.74.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.75.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.76.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.77.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.78.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.79.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.8.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.8.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.8.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.8.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.8.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.8.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.80.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.80.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.80.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.80.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.80.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.80.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.81.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.82.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.83.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.84.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.85.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.86.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.87.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.88.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.89.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.9.down_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.9.down_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.9.gate_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.9.gate_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.9.up_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.9.up_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.mlp.experts.90.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.90.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.90.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.90.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.90.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.90.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.91.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.92.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.93.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.94.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.experts.95.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.gate.e_score_correction_bias": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.gate.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.14.mlp.shared_experts.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.14.post_attention_layernorm.weight": "model-00007-of-00045.safetensors", + "model.layers.14.self_attn.k_norm.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.k_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.k_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.k_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.o_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.o_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.q_norm.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.q_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.q_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.q_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.v_proj.bias": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.v_proj.weight": "model-00006-of-00045.safetensors", + "model.layers.14.self_attn.v_proj.weight_scale": "model-00006-of-00045.safetensors", + "model.layers.15.input_layernorm.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.0.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.1.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.10.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.11.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.12.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.13.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.14.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.15.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.16.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.17.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.18.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.19.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.2.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.20.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.21.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.22.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.23.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.24.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.25.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.26.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.27.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.28.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.29.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.3.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.30.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.31.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.32.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.33.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.34.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.35.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.36.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.37.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.38.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.39.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.4.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.40.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.41.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.42.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.43.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.44.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.45.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.46.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.47.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.48.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.49.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.5.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.50.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.51.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.52.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.53.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.54.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.55.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.56.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.57.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.58.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.59.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.6.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.60.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.61.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.62.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.63.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.64.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.65.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.66.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.67.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.68.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.69.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.7.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.70.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.71.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.72.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.73.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.74.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.75.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.76.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.77.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.78.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.79.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.8.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.80.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.81.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.82.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.83.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.84.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.85.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.86.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.87.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.88.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.89.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.9.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.90.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.91.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.92.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.93.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.94.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.experts.95.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.gate.e_score_correction_bias": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.gate.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.mlp.shared_experts.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.post_attention_layernorm.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.k_norm.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.k_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.k_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.k_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.o_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.o_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.q_norm.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.q_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.q_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.q_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.v_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.v_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.15.self_attn.v_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.input_layernorm.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.0.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.0.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.0.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.0.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.0.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.0.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.1.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.10.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.11.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.12.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.13.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.14.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.15.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.16.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.17.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.18.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.19.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.2.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.20.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.21.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.22.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.23.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.24.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.25.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.26.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.27.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.28.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.29.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.3.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.30.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.30.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.30.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.30.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.30.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.30.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.31.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.31.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.31.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.31.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.31.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.31.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.32.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.33.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.34.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.35.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.36.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.37.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.38.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.39.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.4.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.4.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.4.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.4.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.4.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.4.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.40.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.40.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.40.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.40.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.40.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.40.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.41.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.42.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.43.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.44.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.45.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.46.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.47.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.48.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.49.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.5.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.5.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.5.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.5.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.5.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.5.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.50.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.50.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.50.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.50.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.50.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.50.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.51.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.52.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.53.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.54.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.55.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.56.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.57.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.58.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.59.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.6.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.6.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.6.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.6.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.6.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.6.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.60.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.60.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.60.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.60.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.60.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.60.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.61.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.62.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.63.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.64.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.65.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.66.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.67.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.68.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.69.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.7.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.7.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.7.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.7.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.7.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.7.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.70.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.70.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.70.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.70.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.70.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.70.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.71.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.72.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.73.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.74.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.75.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.76.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.77.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.78.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.79.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.8.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.8.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.8.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.8.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.8.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.8.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.80.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.80.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.80.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.80.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.80.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.80.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.81.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.82.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.83.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.84.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.85.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.86.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.87.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.88.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.89.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.9.down_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.9.down_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.9.gate_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.9.gate_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.9.up_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.9.up_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.mlp.experts.90.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.90.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.90.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.90.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.90.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.90.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.91.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.92.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.93.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.94.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.experts.95.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.gate.e_score_correction_bias": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.gate.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.16.mlp.shared_experts.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.16.post_attention_layernorm.weight": "model-00008-of-00045.safetensors", + "model.layers.16.self_attn.k_norm.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.k_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.k_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.k_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.o_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.o_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.q_norm.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.q_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.q_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.q_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.v_proj.bias": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.v_proj.weight": "model-00007-of-00045.safetensors", + "model.layers.16.self_attn.v_proj.weight_scale": "model-00007-of-00045.safetensors", + "model.layers.17.input_layernorm.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.0.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.1.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.10.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.11.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.12.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.13.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.14.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.15.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.16.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.17.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.18.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.19.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.2.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.20.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.21.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.22.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.23.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.24.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.25.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.26.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.27.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.28.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.29.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.3.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.30.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.31.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.32.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.33.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.34.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.35.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.36.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.37.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.38.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.39.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.4.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.40.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.41.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.42.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.43.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.44.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.45.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.46.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.47.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.48.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.49.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.5.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.50.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.51.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.52.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.53.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.54.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.55.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.56.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.57.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.58.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.59.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.6.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.60.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.61.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.62.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.63.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.64.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.65.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.66.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.67.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.68.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.69.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.7.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.70.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.71.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.72.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.73.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.74.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.75.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.76.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.77.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.78.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.79.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.8.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.80.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.81.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.82.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.83.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.84.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.85.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.86.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.87.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.88.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.89.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.9.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.90.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.91.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.92.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.93.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.94.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.experts.95.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.gate.e_score_correction_bias": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.gate.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.mlp.shared_experts.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.post_attention_layernorm.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.k_norm.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.k_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.k_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.k_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.o_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.o_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.q_norm.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.q_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.q_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.q_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.v_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.v_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.17.self_attn.v_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.input_layernorm.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.0.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.0.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.0.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.0.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.0.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.0.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.1.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.10.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.11.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.12.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.13.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.14.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.15.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.16.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.17.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.18.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.19.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.2.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.20.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.21.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.22.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.23.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.24.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.25.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.26.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.27.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.28.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.29.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.3.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.30.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.31.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.32.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.33.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.34.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.35.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.36.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.36.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.36.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.36.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.36.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.36.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.37.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.37.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.37.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.37.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.37.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.37.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.38.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.39.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.4.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.4.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.4.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.4.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.4.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.4.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.40.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.40.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.40.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.40.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.40.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.40.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.41.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.42.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.43.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.44.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.45.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.46.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.47.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.48.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.49.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.5.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.5.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.5.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.5.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.5.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.5.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.50.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.50.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.50.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.50.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.50.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.50.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.51.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.52.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.53.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.54.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.55.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.56.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.57.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.58.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.59.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.6.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.6.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.6.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.6.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.6.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.6.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.60.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.60.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.60.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.60.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.60.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.60.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.61.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.62.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.63.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.64.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.65.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.66.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.67.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.68.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.69.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.7.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.7.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.7.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.7.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.7.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.7.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.70.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.70.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.70.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.70.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.70.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.70.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.71.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.72.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.73.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.74.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.75.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.76.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.77.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.78.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.79.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.8.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.8.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.8.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.8.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.8.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.8.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.80.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.80.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.80.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.80.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.80.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.80.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.81.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.82.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.83.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.84.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.85.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.86.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.87.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.88.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.89.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.9.down_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.9.down_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.9.gate_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.9.gate_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.9.up_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.9.up_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.mlp.experts.90.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.90.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.90.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.90.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.90.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.90.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.91.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.92.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.93.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.94.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.experts.95.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.gate.e_score_correction_bias": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.gate.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.18.mlp.shared_experts.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.18.post_attention_layernorm.weight": "model-00009-of-00045.safetensors", + "model.layers.18.self_attn.k_norm.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.k_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.k_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.k_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.o_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.o_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.q_norm.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.q_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.q_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.q_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.v_proj.bias": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.v_proj.weight": "model-00008-of-00045.safetensors", + "model.layers.18.self_attn.v_proj.weight_scale": "model-00008-of-00045.safetensors", + "model.layers.19.input_layernorm.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.0.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.1.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.10.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.11.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.12.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.13.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.14.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.15.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.16.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.17.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.18.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.19.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.2.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.20.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.21.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.22.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.23.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.24.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.25.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.26.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.27.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.28.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.29.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.3.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.30.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.31.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.32.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.33.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.34.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.35.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.36.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.37.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.38.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.39.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.4.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.40.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.41.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.42.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.43.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.44.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.45.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.46.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.47.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.48.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.49.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.5.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.50.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.51.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.52.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.53.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.54.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.55.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.56.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.57.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.58.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.59.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.6.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.60.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.61.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.62.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.63.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.64.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.65.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.66.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.67.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.68.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.69.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.7.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.70.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.71.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.72.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.73.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.74.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.75.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.76.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.77.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.78.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.79.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.8.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.80.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.81.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.82.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.83.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.84.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.85.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.86.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.87.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.88.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.89.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.9.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.90.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.91.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.92.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.93.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.94.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.experts.95.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.gate.e_score_correction_bias": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.gate.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.mlp.shared_experts.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.post_attention_layernorm.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.k_norm.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.k_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.k_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.k_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.o_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.o_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.q_norm.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.q_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.q_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.q_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.v_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.v_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.19.self_attn.v_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.2.input_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.mlp.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.k_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.k_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.o_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.q_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.q_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.2.self_attn.v_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.20.input_layernorm.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.0.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.0.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.0.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.0.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.0.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.0.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.1.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.10.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.11.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.12.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.13.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.14.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.15.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.16.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.17.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.18.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.19.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.2.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.20.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.21.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.22.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.23.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.24.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.25.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.26.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.27.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.28.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.29.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.3.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.30.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.31.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.32.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.33.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.34.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.35.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.36.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.37.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.38.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.39.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.4.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.40.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.41.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.42.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.42.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.42.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.42.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.42.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.42.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.43.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.43.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.43.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.43.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.43.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.43.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.44.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.45.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.46.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.47.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.48.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.49.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.5.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.5.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.5.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.5.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.5.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.5.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.50.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.50.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.50.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.50.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.50.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.50.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.51.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.52.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.53.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.54.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.55.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.56.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.57.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.58.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.59.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.6.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.6.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.6.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.6.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.6.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.6.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.60.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.60.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.60.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.60.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.60.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.60.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.61.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.62.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.63.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.64.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.65.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.66.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.67.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.68.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.69.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.7.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.7.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.7.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.7.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.7.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.7.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.70.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.70.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.70.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.70.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.70.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.70.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.71.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.72.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.73.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.74.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.75.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.76.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.77.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.78.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.79.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.8.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.8.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.8.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.8.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.8.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.8.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.80.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.80.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.80.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.80.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.80.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.80.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.81.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.82.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.83.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.84.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.85.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.86.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.87.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.88.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.89.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.9.down_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.9.down_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.9.gate_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.9.gate_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.9.up_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.9.up_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.mlp.experts.90.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.90.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.90.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.90.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.90.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.90.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.91.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.92.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.93.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.94.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.experts.95.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.gate.e_score_correction_bias": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.gate.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.20.mlp.shared_experts.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.20.post_attention_layernorm.weight": "model-00010-of-00045.safetensors", + "model.layers.20.self_attn.k_norm.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.k_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.k_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.k_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.o_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.o_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.q_norm.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.q_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.q_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.q_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.v_proj.bias": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.v_proj.weight": "model-00009-of-00045.safetensors", + "model.layers.20.self_attn.v_proj.weight_scale": "model-00009-of-00045.safetensors", + "model.layers.21.input_layernorm.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.0.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.1.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.10.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.11.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.12.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.13.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.14.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.15.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.16.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.17.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.18.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.19.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.2.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.20.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.21.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.22.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.23.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.24.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.25.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.26.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.27.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.28.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.29.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.3.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.30.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.31.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.32.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.33.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.34.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.35.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.36.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.37.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.38.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.39.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.4.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.40.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.41.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.42.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.43.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.44.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.45.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.46.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.47.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.48.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.49.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.5.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.50.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.51.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.52.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.53.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.54.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.55.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.56.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.57.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.58.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.59.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.6.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.60.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.61.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.62.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.63.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.64.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.65.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.66.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.67.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.68.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.69.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.7.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.70.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.71.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.72.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.73.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.74.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.75.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.76.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.77.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.78.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.79.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.8.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.80.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.81.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.82.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.83.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.84.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.85.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.86.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.87.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.88.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.89.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.9.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.90.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.91.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.92.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.93.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.94.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.experts.95.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.gate.e_score_correction_bias": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.gate.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.mlp.shared_experts.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.post_attention_layernorm.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.k_norm.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.k_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.k_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.k_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.o_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.o_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.q_norm.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.q_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.q_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.q_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.v_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.v_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.21.self_attn.v_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.input_layernorm.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.0.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.0.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.0.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.0.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.0.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.0.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.1.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.10.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.11.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.12.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.13.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.14.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.15.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.16.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.17.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.18.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.19.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.2.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.20.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.21.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.22.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.23.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.24.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.25.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.26.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.27.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.28.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.29.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.3.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.30.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.31.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.32.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.33.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.34.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.35.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.36.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.37.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.38.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.39.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.4.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.40.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.41.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.42.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.43.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.44.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.45.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.46.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.47.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.48.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.48.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.48.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.48.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.48.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.48.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.49.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.49.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.49.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.49.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.49.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.49.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.5.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.5.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.5.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.5.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.5.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.5.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.50.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.50.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.50.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.50.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.50.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.50.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.51.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.52.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.53.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.54.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.55.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.56.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.57.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.58.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.59.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.6.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.6.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.6.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.6.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.6.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.6.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.60.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.60.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.60.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.60.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.60.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.60.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.61.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.62.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.63.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.64.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.65.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.66.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.67.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.68.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.69.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.7.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.7.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.7.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.7.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.7.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.7.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.70.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.70.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.70.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.70.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.70.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.70.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.71.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.72.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.73.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.74.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.75.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.76.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.77.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.78.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.79.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.8.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.8.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.8.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.8.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.8.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.8.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.80.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.80.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.80.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.80.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.80.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.80.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.81.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.82.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.83.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.84.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.85.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.86.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.87.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.88.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.89.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.9.down_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.9.down_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.9.gate_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.9.gate_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.9.up_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.9.up_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.mlp.experts.90.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.90.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.90.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.90.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.90.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.90.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.91.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.92.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.93.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.94.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.experts.95.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.gate.e_score_correction_bias": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.gate.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.22.mlp.shared_experts.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.22.post_attention_layernorm.weight": "model-00011-of-00045.safetensors", + "model.layers.22.self_attn.k_norm.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.k_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.k_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.k_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.o_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.o_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.q_norm.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.q_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.q_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.q_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.v_proj.bias": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.v_proj.weight": "model-00010-of-00045.safetensors", + "model.layers.22.self_attn.v_proj.weight_scale": "model-00010-of-00045.safetensors", + "model.layers.23.input_layernorm.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.0.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.1.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.10.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.11.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.12.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.13.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.14.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.15.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.16.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.17.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.18.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.19.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.2.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.20.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.21.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.22.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.23.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.24.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.25.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.26.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.27.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.28.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.29.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.3.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.30.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.31.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.32.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.33.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.34.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.35.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.36.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.37.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.38.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.39.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.4.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.40.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.41.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.42.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.43.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.44.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.45.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.46.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.47.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.48.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.49.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.5.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.50.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.51.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.52.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.53.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.54.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.55.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.56.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.57.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.58.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.59.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.6.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.60.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.61.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.62.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.63.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.64.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.65.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.66.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.67.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.68.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.69.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.7.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.70.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.71.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.72.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.73.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.74.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.75.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.76.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.77.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.78.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.79.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.8.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.80.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.81.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.82.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.83.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.84.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.85.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.86.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.87.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.88.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.89.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.9.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.90.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.91.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.92.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.93.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.94.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.experts.95.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.gate.e_score_correction_bias": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.gate.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.mlp.shared_experts.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.post_attention_layernorm.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.k_norm.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.k_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.k_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.k_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.o_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.o_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.q_norm.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.q_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.q_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.q_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.v_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.v_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.23.self_attn.v_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.input_layernorm.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.0.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.0.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.0.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.0.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.0.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.0.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.1.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.10.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.11.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.12.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.13.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.14.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.15.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.16.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.17.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.18.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.19.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.2.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.20.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.21.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.22.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.23.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.24.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.25.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.26.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.27.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.28.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.29.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.3.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.30.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.31.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.32.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.33.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.34.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.35.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.36.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.37.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.38.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.39.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.4.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.40.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.41.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.42.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.43.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.44.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.45.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.46.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.47.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.48.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.49.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.5.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.50.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.51.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.52.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.53.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.54.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.54.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.54.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.54.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.54.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.54.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.55.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.55.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.55.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.55.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.55.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.55.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.56.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.57.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.58.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.59.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.6.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.6.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.6.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.6.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.6.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.6.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.60.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.60.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.60.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.60.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.60.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.60.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.61.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.62.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.63.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.64.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.65.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.66.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.67.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.68.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.69.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.7.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.7.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.7.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.7.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.7.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.7.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.70.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.70.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.70.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.70.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.70.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.70.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.71.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.72.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.73.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.74.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.75.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.76.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.77.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.78.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.79.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.8.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.8.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.8.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.8.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.8.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.8.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.80.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.80.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.80.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.80.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.80.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.80.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.81.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.82.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.83.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.84.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.85.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.86.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.87.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.88.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.89.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.9.down_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.9.down_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.9.gate_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.9.gate_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.9.up_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.9.up_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.mlp.experts.90.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.90.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.90.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.90.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.90.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.90.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.91.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.92.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.93.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.94.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.experts.95.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.gate.e_score_correction_bias": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.gate.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.24.mlp.shared_experts.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.24.post_attention_layernorm.weight": "model-00012-of-00045.safetensors", + "model.layers.24.self_attn.k_norm.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.k_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.k_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.k_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.o_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.o_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.q_norm.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.q_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.q_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.q_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.v_proj.bias": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.v_proj.weight": "model-00011-of-00045.safetensors", + "model.layers.24.self_attn.v_proj.weight_scale": "model-00011-of-00045.safetensors", + "model.layers.25.input_layernorm.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.0.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.1.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.10.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.11.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.12.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.13.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.14.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.15.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.16.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.17.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.18.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.19.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.2.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.20.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.21.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.22.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.23.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.24.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.25.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.26.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.27.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.28.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.29.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.3.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.30.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.31.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.32.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.33.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.34.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.35.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.36.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.37.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.38.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.39.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.4.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.40.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.41.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.42.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.43.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.44.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.45.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.46.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.47.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.48.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.49.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.5.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.50.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.51.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.52.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.53.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.54.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.55.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.56.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.57.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.58.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.59.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.6.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.60.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.61.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.62.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.63.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.64.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.65.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.66.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.67.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.68.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.69.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.7.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.70.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.71.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.72.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.73.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.74.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.75.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.76.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.77.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.78.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.79.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.8.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.80.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.81.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.82.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.83.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.84.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.85.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.86.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.87.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.88.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.89.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.9.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.90.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.91.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.92.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.93.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.94.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.experts.95.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.gate.e_score_correction_bias": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.gate.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.mlp.shared_experts.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.post_attention_layernorm.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.k_norm.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.k_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.k_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.k_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.o_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.o_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.q_norm.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.q_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.q_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.q_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.v_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.v_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.25.self_attn.v_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.input_layernorm.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.0.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.0.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.0.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.0.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.0.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.0.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.1.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.10.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.11.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.12.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.13.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.14.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.15.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.16.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.17.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.18.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.19.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.2.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.20.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.21.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.22.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.23.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.24.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.25.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.26.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.27.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.28.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.29.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.3.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.30.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.31.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.32.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.33.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.34.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.35.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.36.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.37.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.38.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.39.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.4.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.40.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.41.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.42.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.43.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.44.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.45.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.46.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.47.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.48.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.49.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.5.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.50.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.51.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.52.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.53.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.54.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.55.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.56.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.57.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.58.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.59.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.6.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.60.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.60.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.60.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.60.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.60.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.60.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.61.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.61.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.61.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.61.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.61.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.61.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.62.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.63.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.64.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.65.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.66.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.67.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.68.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.69.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.7.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.7.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.7.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.7.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.7.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.7.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.70.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.70.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.70.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.70.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.70.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.70.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.71.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.72.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.73.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.74.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.75.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.76.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.77.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.78.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.79.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.8.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.8.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.8.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.8.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.8.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.8.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.80.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.80.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.80.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.80.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.80.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.80.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.81.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.82.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.83.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.84.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.85.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.86.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.87.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.88.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.89.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.9.down_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.9.down_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.9.gate_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.9.gate_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.9.up_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.9.up_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.mlp.experts.90.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.90.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.90.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.90.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.90.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.90.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.91.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.92.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.93.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.94.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.experts.95.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.gate.e_score_correction_bias": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.gate.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.26.mlp.shared_experts.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.26.post_attention_layernorm.weight": "model-00013-of-00045.safetensors", + "model.layers.26.self_attn.k_norm.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.k_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.k_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.k_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.o_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.o_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.q_norm.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.q_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.q_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.q_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.v_proj.bias": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.v_proj.weight": "model-00012-of-00045.safetensors", + "model.layers.26.self_attn.v_proj.weight_scale": "model-00012-of-00045.safetensors", + "model.layers.27.input_layernorm.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.0.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.1.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.10.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.11.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.12.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.13.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.14.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.15.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.16.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.17.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.18.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.19.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.2.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.20.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.21.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.22.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.23.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.24.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.25.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.26.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.27.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.28.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.29.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.3.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.30.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.31.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.32.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.33.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.34.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.35.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.36.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.37.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.38.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.39.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.4.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.40.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.41.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.42.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.43.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.44.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.45.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.46.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.47.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.48.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.49.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.5.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.50.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.51.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.52.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.53.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.54.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.55.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.56.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.57.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.58.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.59.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.6.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.60.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.61.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.62.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.63.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.64.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.65.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.66.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.67.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.68.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.69.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.7.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.70.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.71.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.72.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.73.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.74.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.75.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.76.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.77.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.78.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.79.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.8.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.80.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.81.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.82.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.83.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.84.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.85.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.86.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.87.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.88.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.89.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.9.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.90.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.91.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.92.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.93.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.94.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.experts.95.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.gate.e_score_correction_bias": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.gate.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.mlp.shared_experts.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.post_attention_layernorm.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.k_norm.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.k_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.k_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.k_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.o_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.o_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.q_norm.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.q_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.q_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.q_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.v_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.v_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.27.self_attn.v_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.input_layernorm.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.0.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.0.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.0.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.0.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.0.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.0.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.1.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.10.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.11.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.12.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.13.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.14.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.15.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.16.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.17.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.18.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.19.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.2.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.20.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.21.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.22.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.23.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.24.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.25.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.26.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.27.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.28.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.29.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.3.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.30.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.31.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.32.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.33.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.34.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.35.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.36.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.37.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.38.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.39.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.4.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.40.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.41.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.42.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.43.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.44.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.45.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.46.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.47.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.48.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.49.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.5.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.50.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.51.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.52.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.53.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.54.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.55.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.56.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.57.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.58.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.59.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.6.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.60.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.61.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.62.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.63.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.64.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.65.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.66.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.66.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.66.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.66.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.66.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.66.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.67.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.67.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.67.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.67.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.67.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.67.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.68.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.69.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.7.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.7.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.7.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.7.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.7.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.7.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.70.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.70.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.70.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.70.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.70.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.70.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.71.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.72.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.73.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.74.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.75.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.76.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.77.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.78.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.79.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.8.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.8.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.8.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.8.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.8.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.8.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.80.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.80.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.80.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.80.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.80.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.80.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.81.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.82.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.83.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.84.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.85.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.86.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.87.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.88.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.89.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.9.down_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.9.down_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.9.gate_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.9.gate_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.9.up_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.9.up_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.mlp.experts.90.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.90.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.90.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.90.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.90.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.90.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.91.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.92.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.93.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.94.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.experts.95.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.gate.e_score_correction_bias": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.gate.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.28.mlp.shared_experts.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.28.post_attention_layernorm.weight": "model-00014-of-00045.safetensors", + "model.layers.28.self_attn.k_norm.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.k_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.k_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.k_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.o_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.o_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.q_norm.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.q_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.q_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.q_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.v_proj.bias": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.v_proj.weight": "model-00013-of-00045.safetensors", + "model.layers.28.self_attn.v_proj.weight_scale": "model-00013-of-00045.safetensors", + "model.layers.29.input_layernorm.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.0.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.1.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.10.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.11.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.12.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.13.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.14.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.15.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.16.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.17.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.18.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.19.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.2.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.20.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.21.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.22.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.23.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.24.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.25.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.26.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.27.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.28.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.29.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.3.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.30.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.31.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.32.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.33.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.34.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.35.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.36.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.37.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.38.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.39.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.4.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.40.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.41.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.42.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.43.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.44.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.45.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.46.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.47.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.48.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.49.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.5.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.50.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.51.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.52.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.53.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.54.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.55.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.56.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.57.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.58.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.59.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.6.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.60.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.61.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.62.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.63.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.64.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.65.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.66.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.67.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.68.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.69.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.7.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.70.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.71.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.72.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.73.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.74.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.75.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.76.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.77.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.78.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.79.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.8.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.80.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.81.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.82.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.83.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.84.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.85.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.86.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.87.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.88.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.89.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.9.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.90.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.91.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.92.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.93.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.94.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.experts.95.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.gate.e_score_correction_bias": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.gate.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.mlp.shared_experts.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.post_attention_layernorm.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.k_norm.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.k_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.k_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.k_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.o_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.o_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.q_norm.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.q_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.q_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.q_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.v_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.v_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.29.self_attn.v_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.3.input_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.0.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.1.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.10.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.11.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.12.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.13.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.14.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.15.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.16.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.17.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.18.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.19.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.2.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.20.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.21.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.22.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.23.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.24.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.25.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.26.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.27.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.28.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.29.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.3.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.30.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.31.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.32.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.33.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.34.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.35.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.36.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.37.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.38.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.39.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.4.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.40.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.41.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.42.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.43.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.44.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.45.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.46.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.47.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.48.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.49.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.5.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.50.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.51.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.52.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.53.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.54.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.55.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.56.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.57.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.58.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.59.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.6.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.60.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.61.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.62.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.63.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.64.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.65.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.66.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.67.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.68.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.69.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.7.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.70.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.71.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.72.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.73.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.74.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.75.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.76.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.77.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.78.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.79.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.8.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.80.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.81.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.82.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.83.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.84.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.85.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.86.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.87.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.88.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.89.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.9.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.90.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.91.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.92.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.93.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.94.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.experts.95.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.gate.e_score_correction_bias": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.gate.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.down_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.down_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.gate_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.gate_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.up_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.mlp.shared_experts.up_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.k_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.k_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.o_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.q_norm.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.q_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00045.safetensors", + "model.layers.3.self_attn.v_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.30.input_layernorm.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.0.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.0.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.0.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.0.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.0.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.0.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.1.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.10.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.11.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.12.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.13.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.14.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.15.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.16.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.17.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.18.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.19.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.2.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.20.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.21.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.22.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.23.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.24.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.25.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.26.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.27.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.28.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.29.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.3.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.30.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.31.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.32.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.33.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.34.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.35.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.36.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.37.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.38.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.39.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.4.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.40.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.41.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.42.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.43.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.44.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.45.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.46.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.47.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.48.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.49.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.5.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.50.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.51.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.52.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.53.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.54.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.55.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.56.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.57.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.58.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.59.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.6.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.60.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.61.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.62.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.63.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.64.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.65.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.66.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.67.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.68.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.69.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.7.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.70.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.71.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.72.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.72.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.72.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.72.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.72.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.72.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.73.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.73.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.73.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.73.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.73.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.73.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.74.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.75.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.76.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.77.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.78.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.79.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.8.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.8.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.8.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.8.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.8.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.8.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.80.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.80.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.80.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.80.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.80.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.80.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.81.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.82.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.83.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.84.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.85.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.86.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.87.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.88.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.89.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.9.down_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.9.down_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.9.gate_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.9.gate_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.9.up_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.9.up_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.mlp.experts.90.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.90.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.90.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.90.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.90.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.90.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.91.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.92.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.93.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.94.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.experts.95.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.gate.e_score_correction_bias": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.gate.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.30.mlp.shared_experts.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.30.post_attention_layernorm.weight": "model-00015-of-00045.safetensors", + "model.layers.30.self_attn.k_norm.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.k_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.k_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.k_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.o_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.o_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.q_norm.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.q_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.q_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.q_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.v_proj.bias": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.v_proj.weight": "model-00014-of-00045.safetensors", + "model.layers.30.self_attn.v_proj.weight_scale": "model-00014-of-00045.safetensors", + "model.layers.31.input_layernorm.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.0.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.1.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.10.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.11.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.12.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.13.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.14.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.15.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.16.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.17.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.18.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.19.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.2.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.20.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.21.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.22.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.23.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.24.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.25.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.26.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.27.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.28.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.29.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.3.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.30.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.31.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.32.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.33.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.34.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.35.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.36.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.37.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.38.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.39.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.4.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.40.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.41.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.42.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.43.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.44.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.45.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.46.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.47.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.48.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.49.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.5.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.50.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.51.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.52.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.53.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.54.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.55.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.56.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.57.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.58.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.59.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.6.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.60.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.61.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.62.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.63.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.64.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.65.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.66.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.67.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.68.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.69.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.7.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.70.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.71.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.72.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.73.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.74.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.75.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.76.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.77.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.78.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.79.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.8.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.80.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.81.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.82.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.83.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.84.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.85.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.86.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.87.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.88.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.89.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.9.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.90.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.91.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.92.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.93.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.94.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.experts.95.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.gate.e_score_correction_bias": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.gate.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.mlp.shared_experts.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.post_attention_layernorm.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.k_norm.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.k_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.k_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.k_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.o_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.o_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.q_norm.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.q_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.q_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.q_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.v_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.v_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.31.self_attn.v_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.input_layernorm.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.0.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.0.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.0.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.0.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.0.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.0.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.1.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.10.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.11.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.12.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.13.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.14.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.15.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.16.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.17.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.18.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.19.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.2.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.20.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.21.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.22.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.23.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.24.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.25.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.26.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.27.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.28.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.29.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.3.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.30.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.31.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.32.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.33.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.34.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.35.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.36.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.37.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.38.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.39.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.4.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.40.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.41.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.42.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.43.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.44.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.45.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.46.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.47.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.48.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.49.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.5.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.50.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.51.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.52.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.53.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.54.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.55.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.56.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.57.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.58.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.59.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.6.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.60.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.61.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.62.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.63.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.64.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.65.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.66.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.67.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.68.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.69.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.7.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.70.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.71.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.72.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.73.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.74.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.75.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.76.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.77.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.78.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.78.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.78.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.78.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.78.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.78.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.79.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.79.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.79.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.79.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.79.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.79.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.8.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.8.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.8.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.8.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.8.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.8.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.80.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.80.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.80.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.80.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.80.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.80.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.81.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.82.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.83.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.84.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.85.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.86.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.87.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.88.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.89.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.9.down_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.9.down_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.9.gate_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.9.gate_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.9.up_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.9.up_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.mlp.experts.90.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.90.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.90.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.90.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.90.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.90.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.91.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.92.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.93.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.94.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.experts.95.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.gate.e_score_correction_bias": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.gate.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.32.mlp.shared_experts.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.32.post_attention_layernorm.weight": "model-00016-of-00045.safetensors", + "model.layers.32.self_attn.k_norm.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.k_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.k_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.k_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.o_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.o_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.q_norm.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.q_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.q_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.q_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.v_proj.bias": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.v_proj.weight": "model-00015-of-00045.safetensors", + "model.layers.32.self_attn.v_proj.weight_scale": "model-00015-of-00045.safetensors", + "model.layers.33.input_layernorm.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.0.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.1.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.10.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.11.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.12.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.13.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.14.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.15.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.16.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.17.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.18.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.19.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.2.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.20.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.21.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.22.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.23.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.24.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.25.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.26.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.27.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.28.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.29.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.3.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.30.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.31.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.32.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.33.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.34.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.35.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.36.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.37.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.38.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.39.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.4.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.40.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.41.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.42.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.43.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.44.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.45.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.46.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.47.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.48.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.49.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.5.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.50.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.51.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.52.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.53.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.54.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.55.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.56.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.57.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.58.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.59.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.6.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.60.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.61.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.62.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.63.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.64.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.65.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.66.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.67.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.68.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.69.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.7.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.70.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.71.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.72.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.73.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.74.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.75.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.76.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.77.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.78.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.79.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.8.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.80.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.81.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.82.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.83.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.84.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.85.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.86.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.87.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.88.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.89.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.9.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.90.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.91.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.92.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.93.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.94.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.experts.95.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.gate.e_score_correction_bias": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.gate.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.mlp.shared_experts.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.post_attention_layernorm.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.k_norm.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.k_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.k_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.k_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.o_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.o_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.q_norm.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.q_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.q_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.q_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.v_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.v_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.33.self_attn.v_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.input_layernorm.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.0.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.0.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.0.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.0.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.0.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.0.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.1.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.10.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.11.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.12.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.13.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.14.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.15.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.16.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.17.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.18.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.19.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.2.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.20.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.21.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.22.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.23.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.24.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.25.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.26.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.27.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.28.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.29.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.3.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.30.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.31.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.32.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.33.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.34.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.35.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.36.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.37.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.38.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.39.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.4.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.40.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.41.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.42.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.43.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.44.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.45.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.46.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.47.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.48.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.49.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.5.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.50.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.51.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.52.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.53.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.54.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.55.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.56.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.57.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.58.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.59.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.6.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.60.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.61.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.62.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.63.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.64.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.65.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.66.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.67.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.68.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.69.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.7.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.70.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.71.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.72.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.73.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.74.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.75.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.76.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.77.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.78.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.79.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.8.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.80.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.81.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.82.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.83.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.84.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.84.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.84.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.84.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.84.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.84.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.85.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.85.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.85.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.85.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.85.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.85.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.86.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.87.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.88.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.89.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.9.down_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.9.down_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.9.gate_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.9.gate_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.9.up_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.9.up_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.mlp.experts.90.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.90.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.90.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.90.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.90.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.90.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.91.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.92.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.93.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.94.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.experts.95.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.gate.e_score_correction_bias": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.gate.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.34.mlp.shared_experts.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.34.post_attention_layernorm.weight": "model-00017-of-00045.safetensors", + "model.layers.34.self_attn.k_norm.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.k_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.k_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.k_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.o_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.o_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.q_norm.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.q_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.q_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.q_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.v_proj.bias": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.v_proj.weight": "model-00016-of-00045.safetensors", + "model.layers.34.self_attn.v_proj.weight_scale": "model-00016-of-00045.safetensors", + "model.layers.35.input_layernorm.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.0.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.1.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.10.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.11.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.12.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.13.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.14.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.15.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.16.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.17.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.18.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.19.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.2.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.20.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.21.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.22.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.23.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.24.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.25.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.26.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.27.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.28.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.29.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.3.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.30.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.31.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.32.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.33.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.34.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.35.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.36.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.37.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.38.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.39.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.4.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.40.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.41.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.42.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.43.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.44.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.45.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.46.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.47.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.48.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.49.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.5.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.50.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.51.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.52.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.53.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.54.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.55.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.56.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.57.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.58.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.59.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.6.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.60.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.61.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.62.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.63.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.64.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.65.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.66.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.67.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.68.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.69.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.7.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.70.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.71.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.72.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.73.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.74.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.75.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.76.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.77.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.78.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.79.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.8.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.80.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.81.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.82.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.83.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.84.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.85.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.86.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.87.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.88.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.89.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.9.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.90.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.91.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.92.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.93.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.94.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.experts.95.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.gate.e_score_correction_bias": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.gate.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.mlp.shared_experts.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.post_attention_layernorm.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.k_norm.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.k_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.k_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.k_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.o_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.o_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.q_norm.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.q_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.q_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.q_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.v_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.v_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.35.self_attn.v_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.input_layernorm.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.0.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.0.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.0.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.0.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.0.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.0.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.1.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.10.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.11.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.12.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.13.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.14.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.15.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.16.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.17.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.18.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.19.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.2.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.20.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.21.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.22.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.23.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.24.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.25.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.26.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.27.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.28.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.29.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.3.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.30.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.31.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.32.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.33.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.34.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.35.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.36.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.37.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.38.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.39.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.4.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.40.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.41.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.42.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.43.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.44.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.45.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.46.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.47.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.48.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.49.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.5.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.50.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.51.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.52.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.53.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.54.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.55.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.56.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.57.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.58.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.59.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.6.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.60.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.61.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.62.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.63.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.64.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.65.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.66.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.67.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.68.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.69.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.7.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.70.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.71.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.72.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.73.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.74.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.75.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.76.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.77.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.78.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.79.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.8.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.80.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.81.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.82.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.83.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.84.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.85.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.86.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.87.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.88.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.89.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.down_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.down_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.up_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.9.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.90.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.90.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.90.gate_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.90.gate_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.90.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.90.up_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.mlp.experts.91.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.91.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.91.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.91.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.91.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.91.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.92.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.93.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.94.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.experts.95.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.gate.e_score_correction_bias": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.gate.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.36.mlp.shared_experts.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.36.post_attention_layernorm.weight": "model-00018-of-00045.safetensors", + "model.layers.36.self_attn.k_norm.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.k_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.k_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.k_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.o_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.o_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.q_norm.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.q_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.q_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.q_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.v_proj.bias": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.v_proj.weight": "model-00017-of-00045.safetensors", + "model.layers.36.self_attn.v_proj.weight_scale": "model-00017-of-00045.safetensors", + "model.layers.37.input_layernorm.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.0.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.1.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.10.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.11.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.12.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.13.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.14.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.15.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.16.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.17.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.18.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.19.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.2.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.20.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.21.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.22.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.23.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.24.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.25.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.26.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.27.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.28.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.29.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.3.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.30.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.31.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.32.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.33.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.34.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.35.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.36.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.37.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.38.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.39.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.4.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.40.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.41.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.42.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.43.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.44.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.45.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.46.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.47.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.48.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.49.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.5.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.50.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.51.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.52.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.53.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.54.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.55.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.56.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.57.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.58.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.59.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.6.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.60.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.61.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.62.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.63.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.64.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.65.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.66.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.67.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.68.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.69.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.7.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.70.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.71.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.72.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.73.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.74.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.75.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.76.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.77.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.78.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.79.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.8.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.80.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.81.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.82.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.83.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.84.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.85.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.86.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.87.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.88.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.89.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.9.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.90.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.91.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.92.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.93.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.94.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.experts.95.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.gate.e_score_correction_bias": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.gate.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.mlp.shared_experts.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.post_attention_layernorm.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.k_norm.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.k_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.k_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.k_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.o_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.o_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.q_norm.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.q_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.q_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.q_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.v_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.v_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.37.self_attn.v_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.input_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.38.mlp.experts.0.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.0.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.0.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.0.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.0.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.0.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.1.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.10.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.11.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.12.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.13.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.14.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.15.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.16.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.17.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.18.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.19.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.2.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.20.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.21.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.22.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.23.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.24.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.25.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.26.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.27.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.28.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.29.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.3.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.30.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.31.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.32.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.33.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.34.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.35.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.36.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.37.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.38.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.39.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.4.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.40.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.41.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.42.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.43.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.44.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.45.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.46.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.47.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.48.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.49.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.5.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.50.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.51.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.52.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.53.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.54.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.55.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.56.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.57.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.58.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.59.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.6.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.60.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.61.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.62.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.63.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.64.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.65.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.66.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.67.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.68.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.69.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.7.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.70.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.71.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.72.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.73.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.74.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.75.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.76.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.77.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.78.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.79.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.8.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.80.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.81.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.82.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.83.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.84.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.85.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.86.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.87.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.88.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.89.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.9.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.90.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.91.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.92.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.93.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.94.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.down_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.down_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.up_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.experts.95.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.gate.e_score_correction_bias": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.gate.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.gate_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.gate_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.38.mlp.shared_experts.up_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.post_attention_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.38.self_attn.k_norm.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.k_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.k_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.k_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.o_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.o_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.q_norm.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.q_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.q_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.q_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.v_proj.bias": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.v_proj.weight": "model-00018-of-00045.safetensors", + "model.layers.38.self_attn.v_proj.weight_scale": "model-00018-of-00045.safetensors", + "model.layers.39.input_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.0.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.1.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.10.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.11.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.12.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.13.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.14.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.15.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.16.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.17.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.18.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.19.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.2.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.20.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.21.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.22.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.23.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.24.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.25.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.26.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.27.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.28.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.29.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.3.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.30.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.31.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.32.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.33.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.34.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.35.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.36.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.37.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.38.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.39.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.4.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.40.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.41.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.42.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.43.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.44.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.45.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.46.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.47.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.48.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.49.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.5.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.50.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.51.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.52.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.53.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.54.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.55.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.56.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.57.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.58.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.59.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.6.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.60.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.61.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.62.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.63.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.64.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.65.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.66.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.67.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.68.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.69.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.7.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.70.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.71.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.72.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.73.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.74.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.75.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.76.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.77.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.78.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.79.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.8.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.80.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.81.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.82.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.83.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.84.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.85.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.86.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.87.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.88.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.89.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.9.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.90.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.91.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.92.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.93.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.94.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.experts.95.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.gate.e_score_correction_bias": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.gate.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.mlp.shared_experts.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.post_attention_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.k_norm.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.k_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.k_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.k_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.o_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.o_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.q_norm.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.q_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.q_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.q_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.v_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.v_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.39.self_attn.v_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.4.input_layernorm.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.0.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.1.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.10.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.11.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.12.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.13.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.14.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.15.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.16.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.17.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.18.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.19.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.2.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.20.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.21.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.22.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.23.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.24.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.25.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.26.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.27.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.28.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.29.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.3.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.30.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.31.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.32.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.33.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.34.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.35.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.36.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.37.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.38.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.39.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.4.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.40.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.41.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.42.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.43.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.44.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.45.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.46.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.47.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.48.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.49.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.5.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.50.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.51.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.52.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.53.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.54.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.55.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.56.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.57.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.58.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.59.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.6.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.60.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.61.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.62.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.63.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.64.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.65.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.66.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.67.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.68.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.69.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.7.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.70.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.71.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.72.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.73.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.74.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.75.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.76.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.77.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.78.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.79.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.8.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.80.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.81.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.82.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.83.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.84.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.85.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.86.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.87.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.88.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.89.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.9.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.90.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.91.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.92.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.93.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.94.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.experts.95.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.gate.e_score_correction_bias": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.gate.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.mlp.shared_experts.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.k_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.k_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.k_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.o_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.q_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.q_proj.bias": "model-00001-of-00045.safetensors", + "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.q_proj.weight_scale": "model-00001-of-00045.safetensors", + "model.layers.4.self_attn.v_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.4.self_attn.v_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.40.input_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.0.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.1.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.10.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.11.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.12.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.13.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.14.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.15.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.16.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.17.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.18.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.19.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.2.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.20.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.21.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.22.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.23.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.24.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.25.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.26.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.27.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.28.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.29.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.3.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.30.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.31.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.32.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.33.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.34.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.35.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.36.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.37.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.38.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.39.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.4.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.40.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.41.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.42.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.43.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.44.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.45.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.46.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.47.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.48.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.49.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.5.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.50.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.51.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.52.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.53.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.54.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.55.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.56.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.57.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.58.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.59.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.6.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.60.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.61.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.62.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.63.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.64.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.65.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.66.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.67.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.68.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.69.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.7.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.70.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.71.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.72.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.73.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.74.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.75.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.76.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.77.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.78.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.79.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.8.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.80.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.81.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.82.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.83.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.84.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.85.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.86.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.87.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.88.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.89.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.9.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.90.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.91.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.92.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.93.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.94.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.experts.95.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.gate.e_score_correction_bias": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.gate.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.down_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.down_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.gate_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.gate_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.up_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.mlp.shared_experts.up_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.post_attention_layernorm.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.k_norm.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.k_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.k_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.k_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.o_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.o_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.q_norm.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.q_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.q_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.q_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.v_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.v_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.40.self_attn.v_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.41.input_layernorm.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.0.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.1.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.10.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.11.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.12.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.13.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.14.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.15.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.16.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.17.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.18.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.19.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.2.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.20.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.21.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.22.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.23.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.24.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.25.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.26.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.27.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.28.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.29.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.3.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.30.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.31.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.32.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.33.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.34.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.35.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.36.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.37.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.38.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.39.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.4.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.40.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.41.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.42.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.43.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.44.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.45.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.46.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.47.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.48.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.49.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.5.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.50.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.51.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.52.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.53.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.54.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.55.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.56.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.57.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.58.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.59.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.6.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.60.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.61.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.62.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.63.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.64.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.65.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.66.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.67.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.68.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.69.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.7.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.70.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.71.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.72.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.73.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.74.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.75.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.76.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.77.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.78.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.79.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.8.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.80.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.81.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.82.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.83.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.84.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.85.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.86.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.87.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.88.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.89.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.9.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.90.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.91.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.92.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.93.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.94.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.experts.95.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.gate.e_score_correction_bias": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.gate.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.mlp.shared_experts.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.41.post_attention_layernorm.weight": "model-00020-of-00045.safetensors", + "model.layers.41.self_attn.k_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.41.self_attn.k_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.k_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.k_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.o_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.41.self_attn.o_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.q_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.41.self_attn.q_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.q_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.q_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.v_proj.bias": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.v_proj.weight": "model-00019-of-00045.safetensors", + "model.layers.41.self_attn.v_proj.weight_scale": "model-00019-of-00045.safetensors", + "model.layers.42.input_layernorm.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.0.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.1.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.10.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.11.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.12.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.13.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.14.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.15.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.16.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.17.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.18.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.19.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.2.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.20.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.21.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.22.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.23.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.24.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.25.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.26.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.27.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.28.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.29.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.3.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.30.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.31.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.32.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.33.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.34.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.35.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.36.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.37.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.38.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.39.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.4.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.40.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.41.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.42.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.43.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.44.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.45.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.46.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.47.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.48.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.49.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.5.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.50.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.51.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.52.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.53.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.54.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.55.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.56.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.57.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.58.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.59.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.6.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.60.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.61.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.62.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.63.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.64.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.65.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.66.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.67.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.68.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.69.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.7.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.70.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.71.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.72.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.73.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.74.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.75.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.76.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.77.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.78.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.79.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.8.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.80.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.81.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.82.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.83.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.84.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.85.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.86.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.87.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.88.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.89.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.9.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.90.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.91.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.92.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.93.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.94.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.experts.95.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.gate.e_score_correction_bias": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.gate.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.mlp.shared_experts.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.post_attention_layernorm.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.k_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.k_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.k_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.k_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.o_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.o_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.q_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.q_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.q_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.q_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.v_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.v_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.42.self_attn.v_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.input_layernorm.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.0.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.0.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.0.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.0.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.0.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.0.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.1.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.10.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.10.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.10.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.10.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.10.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.10.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.11.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.12.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.13.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.14.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.15.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.16.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.17.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.18.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.19.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.2.down_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.2.down_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.2.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.2.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.2.up_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.2.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.20.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.20.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.20.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.20.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.20.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.20.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.21.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.22.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.23.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.24.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.25.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.26.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.27.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.28.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.29.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.3.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.3.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.3.gate_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.3.gate_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.3.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.3.up_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.mlp.experts.30.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.30.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.30.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.30.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.30.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.30.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.31.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.32.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.33.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.34.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.35.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.36.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.37.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.38.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.39.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.4.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.40.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.41.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.42.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.43.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.44.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.45.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.46.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.47.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.48.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.49.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.5.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.50.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.51.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.52.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.53.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.54.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.55.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.56.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.57.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.58.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.59.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.6.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.60.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.61.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.62.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.63.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.64.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.65.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.66.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.67.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.68.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.69.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.7.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.70.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.71.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.72.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.73.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.74.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.75.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.76.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.77.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.78.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.79.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.8.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.80.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.81.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.82.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.83.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.84.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.85.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.86.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.87.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.88.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.89.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.9.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.90.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.91.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.92.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.93.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.94.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.experts.95.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.gate.e_score_correction_bias": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.gate.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.43.mlp.shared_experts.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.43.post_attention_layernorm.weight": "model-00021-of-00045.safetensors", + "model.layers.43.self_attn.k_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.k_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.k_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.k_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.o_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.o_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.q_norm.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.q_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.q_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.q_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.v_proj.bias": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.v_proj.weight": "model-00020-of-00045.safetensors", + "model.layers.43.self_attn.v_proj.weight_scale": "model-00020-of-00045.safetensors", + "model.layers.44.input_layernorm.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.0.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.1.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.10.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.11.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.12.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.13.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.14.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.15.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.16.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.17.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.18.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.19.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.2.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.20.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.21.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.22.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.23.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.24.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.25.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.26.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.27.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.28.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.29.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.3.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.30.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.31.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.32.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.33.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.34.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.35.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.36.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.37.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.38.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.39.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.4.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.40.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.41.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.42.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.43.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.44.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.45.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.46.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.47.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.48.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.49.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.5.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.50.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.51.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.52.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.53.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.54.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.55.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.56.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.57.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.58.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.59.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.6.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.60.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.61.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.62.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.63.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.64.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.65.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.66.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.67.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.68.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.69.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.7.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.70.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.71.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.72.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.73.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.74.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.75.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.76.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.77.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.78.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.79.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.8.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.80.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.81.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.82.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.83.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.84.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.85.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.86.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.87.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.88.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.89.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.9.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.90.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.91.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.92.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.93.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.94.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.experts.95.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.gate.e_score_correction_bias": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.gate.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.mlp.shared_experts.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.post_attention_layernorm.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.k_norm.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.k_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.k_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.k_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.o_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.o_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.q_norm.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.q_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.q_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.q_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.v_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.v_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.44.self_attn.v_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.input_layernorm.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.0.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.0.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.0.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.0.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.0.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.0.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.1.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.10.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.10.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.10.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.10.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.10.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.10.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.11.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.12.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.13.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.14.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.15.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.16.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.17.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.18.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.19.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.2.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.2.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.2.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.2.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.2.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.2.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.20.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.20.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.20.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.20.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.20.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.20.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.21.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.22.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.23.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.24.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.25.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.26.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.27.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.28.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.29.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.3.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.3.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.3.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.3.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.3.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.3.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.30.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.30.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.30.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.30.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.30.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.30.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.31.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.32.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.33.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.34.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.35.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.36.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.37.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.38.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.39.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.4.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.4.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.4.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.4.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.4.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.4.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.40.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.40.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.40.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.40.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.40.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.40.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.41.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.42.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.43.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.44.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.45.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.46.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.47.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.48.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.49.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.5.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.5.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.5.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.5.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.5.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.5.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.50.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.50.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.50.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.50.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.50.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.50.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.51.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.52.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.53.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.54.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.55.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.56.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.57.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.58.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.59.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.6.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.6.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.6.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.6.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.6.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.6.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.60.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.60.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.60.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.60.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.60.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.60.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.61.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.62.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.63.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.64.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.65.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.66.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.67.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.68.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.69.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.7.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.7.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.7.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.7.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.7.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.7.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.70.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.70.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.70.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.70.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.70.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.70.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.71.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.72.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.73.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.74.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.75.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.76.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.77.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.78.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.79.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.8.down_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.8.down_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.8.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.8.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.8.up_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.8.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.80.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.80.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.80.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.80.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.80.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.80.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.81.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.82.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.83.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.84.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.85.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.86.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.87.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.88.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.89.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.9.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.9.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.9.gate_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.9.gate_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.9.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.9.up_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.mlp.experts.90.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.90.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.90.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.90.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.90.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.90.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.91.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.92.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.93.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.94.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.experts.95.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.gate.e_score_correction_bias": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.gate.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.45.mlp.shared_experts.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.45.post_attention_layernorm.weight": "model-00022-of-00045.safetensors", + "model.layers.45.self_attn.k_norm.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.k_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.k_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.k_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.o_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.o_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.q_norm.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.q_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.q_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.q_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.v_proj.bias": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.v_proj.weight": "model-00021-of-00045.safetensors", + "model.layers.45.self_attn.v_proj.weight_scale": "model-00021-of-00045.safetensors", + "model.layers.46.input_layernorm.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.0.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.1.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.10.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.11.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.12.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.13.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.14.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.15.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.16.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.17.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.18.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.19.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.2.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.20.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.21.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.22.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.23.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.24.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.25.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.26.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.27.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.28.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.29.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.3.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.30.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.31.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.32.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.33.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.34.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.35.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.36.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.37.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.38.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.39.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.4.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.40.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.41.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.42.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.43.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.44.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.45.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.46.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.47.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.48.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.49.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.5.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.50.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.51.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.52.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.53.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.54.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.55.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.56.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.57.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.58.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.59.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.6.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.60.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.61.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.62.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.63.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.64.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.65.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.66.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.67.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.68.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.69.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.7.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.70.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.71.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.72.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.73.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.74.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.75.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.76.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.77.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.78.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.79.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.8.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.80.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.81.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.82.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.83.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.84.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.85.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.86.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.87.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.88.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.89.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.9.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.90.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.91.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.92.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.93.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.94.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.experts.95.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.gate.e_score_correction_bias": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.gate.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.mlp.shared_experts.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.post_attention_layernorm.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.k_norm.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.k_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.k_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.k_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.o_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.o_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.q_norm.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.q_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.q_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.q_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.v_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.v_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.46.self_attn.v_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.input_layernorm.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.0.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.0.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.0.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.0.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.0.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.0.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.1.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.10.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.11.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.12.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.13.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.14.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.15.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.15.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.15.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.15.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.15.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.15.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.16.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.16.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.16.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.16.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.16.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.16.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.17.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.18.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.19.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.2.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.2.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.2.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.2.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.2.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.2.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.20.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.20.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.20.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.20.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.20.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.20.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.21.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.22.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.23.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.24.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.25.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.26.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.27.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.28.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.29.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.3.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.3.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.3.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.3.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.3.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.3.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.30.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.30.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.30.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.30.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.30.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.30.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.31.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.32.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.33.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.34.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.35.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.36.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.37.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.38.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.39.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.4.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.4.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.4.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.4.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.4.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.4.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.40.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.40.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.40.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.40.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.40.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.40.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.41.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.42.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.43.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.44.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.45.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.46.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.47.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.48.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.49.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.5.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.5.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.5.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.5.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.5.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.5.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.50.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.50.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.50.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.50.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.50.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.50.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.51.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.52.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.53.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.54.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.55.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.56.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.57.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.58.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.59.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.6.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.6.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.6.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.6.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.6.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.6.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.60.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.60.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.60.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.60.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.60.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.60.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.61.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.62.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.63.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.64.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.65.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.66.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.67.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.68.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.69.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.7.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.7.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.7.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.7.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.7.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.7.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.70.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.70.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.70.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.70.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.70.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.70.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.71.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.72.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.73.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.74.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.75.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.76.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.77.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.78.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.79.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.8.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.8.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.8.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.8.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.8.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.8.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.80.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.80.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.80.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.80.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.80.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.80.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.81.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.82.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.83.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.84.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.85.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.86.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.87.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.88.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.89.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.9.down_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.9.down_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.9.gate_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.9.gate_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.9.up_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.9.up_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.mlp.experts.90.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.90.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.90.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.90.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.90.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.90.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.91.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.92.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.93.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.94.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.experts.95.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.gate.e_score_correction_bias": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.gate.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.47.mlp.shared_experts.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.47.post_attention_layernorm.weight": "model-00023-of-00045.safetensors", + "model.layers.47.self_attn.k_norm.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.k_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.k_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.k_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.o_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.o_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.q_norm.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.q_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.q_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.q_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.v_proj.bias": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.v_proj.weight": "model-00022-of-00045.safetensors", + "model.layers.47.self_attn.v_proj.weight_scale": "model-00022-of-00045.safetensors", + "model.layers.48.input_layernorm.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.0.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.1.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.10.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.11.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.12.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.13.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.14.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.15.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.16.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.17.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.18.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.19.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.2.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.20.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.21.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.22.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.23.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.24.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.25.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.26.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.27.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.28.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.29.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.3.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.30.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.31.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.32.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.33.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.34.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.35.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.36.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.37.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.38.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.39.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.4.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.40.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.41.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.42.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.43.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.44.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.45.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.46.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.47.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.48.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.49.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.5.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.50.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.51.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.52.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.53.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.54.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.55.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.56.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.57.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.58.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.59.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.6.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.60.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.61.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.62.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.63.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.64.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.65.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.66.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.67.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.68.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.69.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.7.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.70.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.71.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.72.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.73.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.74.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.75.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.76.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.77.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.78.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.79.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.8.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.80.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.81.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.82.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.83.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.84.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.85.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.86.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.87.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.88.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.89.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.9.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.90.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.91.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.92.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.93.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.94.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.experts.95.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.gate.e_score_correction_bias": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.gate.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.mlp.shared_experts.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.post_attention_layernorm.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.k_norm.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.k_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.k_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.k_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.o_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.o_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.q_norm.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.q_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.q_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.q_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.v_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.v_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.48.self_attn.v_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.input_layernorm.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.0.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.0.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.0.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.0.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.0.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.0.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.1.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.10.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.11.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.12.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.13.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.14.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.15.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.16.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.17.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.18.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.19.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.2.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.20.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.21.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.21.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.21.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.21.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.21.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.21.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.22.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.22.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.22.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.22.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.22.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.22.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.23.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.24.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.25.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.26.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.27.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.28.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.29.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.3.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.3.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.3.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.3.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.3.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.3.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.30.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.30.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.30.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.30.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.30.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.30.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.31.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.32.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.33.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.34.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.35.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.36.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.37.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.38.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.39.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.4.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.4.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.4.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.4.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.4.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.4.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.40.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.40.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.40.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.40.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.40.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.40.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.41.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.42.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.43.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.44.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.45.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.46.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.47.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.48.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.49.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.5.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.5.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.5.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.5.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.5.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.5.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.50.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.50.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.50.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.50.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.50.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.50.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.51.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.52.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.53.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.54.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.55.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.56.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.57.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.58.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.59.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.6.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.6.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.6.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.6.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.6.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.6.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.60.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.60.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.60.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.60.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.60.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.60.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.61.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.62.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.63.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.64.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.65.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.66.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.67.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.68.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.69.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.7.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.7.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.7.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.7.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.7.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.7.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.70.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.70.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.70.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.70.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.70.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.70.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.71.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.72.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.73.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.74.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.75.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.76.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.77.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.78.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.79.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.8.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.8.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.8.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.8.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.8.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.8.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.80.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.80.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.80.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.80.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.80.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.80.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.81.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.82.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.83.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.84.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.85.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.86.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.87.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.88.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.89.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.9.down_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.9.down_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.9.gate_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.9.gate_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.9.up_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.9.up_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.mlp.experts.90.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.90.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.90.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.90.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.90.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.90.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.91.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.92.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.93.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.94.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.experts.95.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.gate.e_score_correction_bias": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.gate.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.49.mlp.shared_experts.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.49.post_attention_layernorm.weight": "model-00024-of-00045.safetensors", + "model.layers.49.self_attn.k_norm.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.k_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.k_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.k_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.o_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.o_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.q_norm.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.q_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.q_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.q_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.v_proj.bias": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.v_proj.weight": "model-00023-of-00045.safetensors", + "model.layers.49.self_attn.v_proj.weight_scale": "model-00023-of-00045.safetensors", + "model.layers.5.input_layernorm.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.0.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.1.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.10.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.11.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.12.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.13.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.14.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.15.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.16.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.17.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.18.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.19.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.2.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.20.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.21.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.22.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.23.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.24.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.25.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.26.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.27.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.28.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.29.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.3.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.30.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.31.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.32.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.33.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.34.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.35.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.36.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.37.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.38.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.39.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.4.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.40.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.41.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.42.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.43.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.44.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.45.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.46.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.47.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.48.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.49.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.5.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.50.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.51.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.52.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.53.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.54.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.55.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.56.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.57.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.58.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.59.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.6.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.60.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.61.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.62.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.63.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.64.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.65.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.66.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.67.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.68.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.69.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.7.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.70.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.71.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.72.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.73.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.74.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.75.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.76.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.77.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.78.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.79.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.8.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.80.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.81.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.82.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.83.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.84.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.85.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.86.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.87.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.88.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.89.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.9.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.90.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.91.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.92.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.93.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.94.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.experts.95.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.gate.e_score_correction_bias": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.gate.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.down_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.down_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.up_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.mlp.shared_experts.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.k_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.k_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.k_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.o_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.q_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.q_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.q_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.v_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.5.self_attn.v_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.50.input_layernorm.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.0.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.1.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.10.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.11.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.12.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.13.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.14.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.15.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.16.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.17.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.18.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.19.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.2.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.20.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.21.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.22.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.23.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.24.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.25.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.26.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.27.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.28.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.29.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.3.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.30.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.31.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.32.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.33.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.34.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.35.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.36.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.37.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.38.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.39.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.4.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.40.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.41.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.42.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.43.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.44.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.45.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.46.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.47.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.48.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.49.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.5.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.50.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.51.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.52.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.53.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.54.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.55.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.56.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.57.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.58.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.59.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.6.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.60.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.61.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.62.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.63.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.64.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.65.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.66.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.67.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.68.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.69.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.7.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.70.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.71.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.72.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.73.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.74.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.75.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.76.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.77.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.78.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.79.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.8.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.80.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.81.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.82.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.83.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.84.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.85.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.86.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.87.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.88.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.89.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.9.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.90.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.91.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.92.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.93.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.94.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.experts.95.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.gate.e_score_correction_bias": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.gate.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.mlp.shared_experts.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.post_attention_layernorm.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.k_norm.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.k_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.k_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.k_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.o_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.o_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.q_norm.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.q_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.q_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.q_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.v_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.v_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.50.self_attn.v_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.input_layernorm.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.0.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.0.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.0.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.0.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.0.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.0.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.1.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.10.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.11.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.12.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.13.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.14.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.15.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.16.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.17.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.18.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.19.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.2.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.20.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.21.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.22.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.23.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.24.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.25.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.26.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.27.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.27.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.27.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.27.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.27.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.27.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.28.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.28.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.28.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.28.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.28.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.28.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.29.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.3.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.3.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.3.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.3.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.3.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.3.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.30.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.30.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.30.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.30.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.30.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.30.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.31.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.32.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.33.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.34.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.35.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.36.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.37.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.38.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.39.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.4.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.4.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.4.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.4.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.4.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.4.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.40.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.40.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.40.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.40.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.40.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.40.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.41.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.42.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.43.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.44.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.45.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.46.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.47.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.48.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.49.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.5.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.5.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.5.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.5.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.5.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.5.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.50.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.50.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.50.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.50.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.50.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.50.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.51.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.52.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.53.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.54.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.55.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.56.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.57.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.58.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.59.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.6.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.6.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.6.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.6.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.6.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.6.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.60.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.60.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.60.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.60.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.60.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.60.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.61.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.62.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.63.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.64.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.65.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.66.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.67.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.68.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.69.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.7.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.7.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.7.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.7.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.7.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.7.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.70.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.70.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.70.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.70.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.70.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.70.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.71.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.72.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.73.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.74.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.75.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.76.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.77.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.78.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.79.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.8.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.8.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.8.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.8.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.8.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.8.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.80.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.80.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.80.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.80.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.80.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.80.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.81.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.82.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.83.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.84.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.85.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.86.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.87.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.88.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.89.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.9.down_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.9.down_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.9.gate_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.9.gate_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.9.up_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.9.up_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.mlp.experts.90.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.90.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.90.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.90.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.90.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.90.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.91.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.92.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.93.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.94.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.experts.95.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.gate.e_score_correction_bias": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.gate.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.51.mlp.shared_experts.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.51.post_attention_layernorm.weight": "model-00025-of-00045.safetensors", + "model.layers.51.self_attn.k_norm.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.k_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.k_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.k_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.o_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.o_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.q_norm.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.q_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.q_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.q_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.v_proj.bias": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.v_proj.weight": "model-00024-of-00045.safetensors", + "model.layers.51.self_attn.v_proj.weight_scale": "model-00024-of-00045.safetensors", + "model.layers.52.input_layernorm.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.0.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.1.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.10.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.11.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.12.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.13.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.14.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.15.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.16.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.17.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.18.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.19.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.2.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.20.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.21.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.22.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.23.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.24.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.25.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.26.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.27.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.28.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.29.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.3.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.30.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.31.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.32.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.33.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.34.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.35.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.36.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.37.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.38.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.39.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.4.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.40.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.41.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.42.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.43.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.44.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.45.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.46.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.47.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.48.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.49.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.5.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.50.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.51.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.52.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.53.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.54.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.55.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.56.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.57.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.58.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.59.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.6.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.60.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.61.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.62.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.63.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.64.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.65.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.66.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.67.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.68.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.69.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.7.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.70.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.71.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.72.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.73.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.74.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.75.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.76.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.77.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.78.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.79.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.8.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.80.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.81.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.82.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.83.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.84.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.85.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.86.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.87.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.88.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.89.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.9.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.90.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.91.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.92.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.93.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.94.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.experts.95.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.gate.e_score_correction_bias": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.gate.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.mlp.shared_experts.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.post_attention_layernorm.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.k_norm.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.k_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.k_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.k_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.o_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.o_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.q_norm.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.q_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.q_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.q_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.v_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.v_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.52.self_attn.v_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.input_layernorm.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.0.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.0.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.0.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.0.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.0.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.0.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.1.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.10.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.11.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.12.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.13.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.14.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.15.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.16.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.17.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.18.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.19.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.2.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.20.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.21.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.22.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.23.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.24.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.25.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.26.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.27.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.28.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.29.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.3.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.30.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.31.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.32.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.33.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.33.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.33.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.33.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.33.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.33.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.34.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.34.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.34.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.34.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.34.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.34.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.35.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.36.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.37.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.38.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.39.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.4.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.4.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.4.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.4.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.4.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.4.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.40.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.40.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.40.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.40.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.40.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.40.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.41.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.42.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.43.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.44.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.45.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.46.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.47.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.48.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.49.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.5.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.5.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.5.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.5.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.5.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.5.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.50.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.50.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.50.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.50.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.50.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.50.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.51.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.52.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.53.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.54.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.55.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.56.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.57.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.58.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.59.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.6.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.6.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.6.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.6.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.6.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.6.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.60.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.60.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.60.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.60.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.60.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.60.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.61.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.62.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.63.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.64.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.65.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.66.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.67.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.68.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.69.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.7.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.7.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.7.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.7.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.7.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.7.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.70.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.70.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.70.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.70.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.70.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.70.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.71.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.72.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.73.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.74.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.75.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.76.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.77.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.78.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.79.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.8.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.8.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.8.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.8.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.8.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.8.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.80.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.80.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.80.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.80.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.80.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.80.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.81.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.82.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.83.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.84.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.85.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.86.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.87.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.88.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.89.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.9.down_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.9.down_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.9.gate_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.9.gate_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.9.up_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.9.up_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.mlp.experts.90.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.90.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.90.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.90.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.90.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.90.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.91.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.92.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.93.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.94.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.experts.95.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.gate.e_score_correction_bias": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.gate.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.53.mlp.shared_experts.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.53.post_attention_layernorm.weight": "model-00026-of-00045.safetensors", + "model.layers.53.self_attn.k_norm.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.k_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.k_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.k_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.o_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.o_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.q_norm.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.q_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.q_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.q_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.v_proj.bias": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.v_proj.weight": "model-00025-of-00045.safetensors", + "model.layers.53.self_attn.v_proj.weight_scale": "model-00025-of-00045.safetensors", + "model.layers.54.input_layernorm.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.0.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.1.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.10.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.11.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.12.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.13.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.14.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.15.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.16.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.17.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.18.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.19.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.2.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.20.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.21.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.22.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.23.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.24.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.25.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.26.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.27.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.28.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.29.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.3.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.30.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.31.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.32.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.33.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.34.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.35.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.36.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.37.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.38.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.39.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.4.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.40.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.41.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.42.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.43.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.44.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.45.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.46.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.47.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.48.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.49.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.5.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.50.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.51.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.52.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.53.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.54.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.55.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.56.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.57.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.58.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.59.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.6.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.60.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.61.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.62.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.63.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.64.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.65.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.66.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.67.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.68.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.69.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.7.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.70.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.71.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.72.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.73.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.74.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.75.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.76.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.77.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.78.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.79.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.8.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.80.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.81.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.82.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.83.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.84.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.85.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.86.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.87.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.88.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.89.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.9.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.90.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.91.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.92.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.93.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.94.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.experts.95.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.gate.e_score_correction_bias": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.gate.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.mlp.shared_experts.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.post_attention_layernorm.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.k_norm.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.k_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.k_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.k_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.o_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.o_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.q_norm.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.q_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.q_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.q_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.v_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.v_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.54.self_attn.v_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.input_layernorm.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.0.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.0.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.0.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.0.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.0.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.0.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.1.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.10.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.11.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.12.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.13.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.14.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.15.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.16.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.17.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.18.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.19.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.2.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.20.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.21.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.22.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.23.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.24.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.25.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.26.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.27.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.28.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.29.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.3.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.30.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.31.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.32.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.33.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.34.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.35.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.36.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.37.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.38.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.39.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.39.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.39.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.39.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.39.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.39.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.4.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.40.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.40.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.40.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.40.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.40.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.40.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.41.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.42.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.43.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.44.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.45.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.46.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.47.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.48.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.49.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.5.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.5.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.5.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.5.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.5.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.5.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.50.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.50.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.50.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.50.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.50.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.50.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.51.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.52.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.53.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.54.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.55.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.56.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.57.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.58.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.59.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.6.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.6.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.6.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.6.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.6.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.6.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.60.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.60.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.60.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.60.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.60.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.60.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.61.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.62.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.63.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.64.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.65.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.66.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.67.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.68.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.69.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.7.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.7.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.7.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.7.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.7.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.7.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.70.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.70.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.70.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.70.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.70.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.70.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.71.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.72.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.73.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.74.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.75.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.76.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.77.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.78.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.79.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.8.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.8.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.8.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.8.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.8.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.8.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.80.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.80.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.80.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.80.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.80.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.80.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.81.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.82.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.83.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.84.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.85.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.86.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.87.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.88.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.89.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.9.down_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.9.down_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.9.gate_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.9.gate_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.9.up_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.9.up_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.mlp.experts.90.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.90.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.90.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.90.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.90.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.90.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.91.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.92.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.93.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.94.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.experts.95.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.gate.e_score_correction_bias": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.gate.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.55.mlp.shared_experts.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.55.post_attention_layernorm.weight": "model-00027-of-00045.safetensors", + "model.layers.55.self_attn.k_norm.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.k_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.k_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.k_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.o_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.o_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.q_norm.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.q_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.q_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.q_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.v_proj.bias": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.v_proj.weight": "model-00026-of-00045.safetensors", + "model.layers.55.self_attn.v_proj.weight_scale": "model-00026-of-00045.safetensors", + "model.layers.56.input_layernorm.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.0.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.1.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.10.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.11.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.12.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.13.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.14.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.15.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.16.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.17.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.18.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.19.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.2.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.20.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.21.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.22.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.23.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.24.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.25.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.26.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.27.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.28.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.29.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.3.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.30.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.31.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.32.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.33.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.34.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.35.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.36.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.37.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.38.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.39.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.4.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.40.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.41.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.42.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.43.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.44.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.45.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.46.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.47.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.48.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.49.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.5.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.50.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.51.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.52.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.53.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.54.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.55.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.56.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.57.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.58.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.59.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.6.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.60.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.61.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.62.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.63.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.64.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.65.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.66.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.67.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.68.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.69.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.7.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.70.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.71.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.72.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.73.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.74.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.75.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.76.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.77.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.78.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.79.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.8.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.80.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.81.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.82.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.83.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.84.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.85.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.86.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.87.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.88.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.89.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.9.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.90.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.91.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.92.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.93.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.94.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.experts.95.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.gate.e_score_correction_bias": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.gate.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.mlp.shared_experts.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.post_attention_layernorm.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.k_norm.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.k_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.k_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.k_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.o_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.o_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.q_norm.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.q_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.q_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.q_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.v_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.v_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.56.self_attn.v_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.input_layernorm.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.0.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.0.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.0.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.0.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.0.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.0.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.1.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.10.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.11.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.12.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.13.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.14.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.15.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.16.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.17.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.18.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.19.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.2.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.20.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.21.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.22.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.23.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.24.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.25.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.26.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.27.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.28.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.29.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.3.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.30.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.31.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.32.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.33.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.34.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.35.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.36.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.37.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.38.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.39.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.4.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.40.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.41.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.42.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.43.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.44.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.45.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.45.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.45.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.45.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.45.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.45.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.46.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.46.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.46.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.46.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.46.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.46.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.47.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.48.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.49.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.5.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.5.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.5.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.5.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.5.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.5.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.50.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.50.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.50.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.50.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.50.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.50.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.51.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.52.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.53.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.54.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.55.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.56.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.57.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.58.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.59.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.6.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.6.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.6.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.6.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.6.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.6.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.60.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.60.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.60.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.60.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.60.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.60.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.61.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.62.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.63.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.64.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.65.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.66.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.67.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.68.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.69.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.7.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.7.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.7.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.7.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.7.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.7.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.70.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.70.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.70.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.70.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.70.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.70.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.71.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.72.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.73.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.74.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.75.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.76.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.77.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.78.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.79.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.8.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.8.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.8.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.8.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.8.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.8.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.80.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.80.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.80.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.80.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.80.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.80.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.81.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.82.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.83.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.84.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.85.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.86.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.87.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.88.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.89.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.9.down_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.9.down_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.9.gate_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.9.gate_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.9.up_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.9.up_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.mlp.experts.90.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.90.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.90.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.90.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.90.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.90.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.91.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.92.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.93.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.94.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.experts.95.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.gate.e_score_correction_bias": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.gate.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.57.mlp.shared_experts.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.57.post_attention_layernorm.weight": "model-00028-of-00045.safetensors", + "model.layers.57.self_attn.k_norm.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.k_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.k_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.k_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.o_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.o_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.q_norm.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.q_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.q_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.q_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.v_proj.bias": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.v_proj.weight": "model-00027-of-00045.safetensors", + "model.layers.57.self_attn.v_proj.weight_scale": "model-00027-of-00045.safetensors", + "model.layers.58.input_layernorm.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.0.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.1.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.10.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.11.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.12.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.13.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.14.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.15.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.16.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.17.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.18.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.19.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.2.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.20.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.21.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.22.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.23.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.24.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.25.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.26.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.27.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.28.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.29.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.3.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.30.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.31.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.32.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.33.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.34.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.35.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.36.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.37.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.38.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.39.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.4.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.40.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.41.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.42.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.43.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.44.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.45.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.46.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.47.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.48.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.49.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.5.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.50.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.51.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.52.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.53.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.54.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.55.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.56.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.57.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.58.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.59.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.6.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.60.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.61.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.62.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.63.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.64.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.65.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.66.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.67.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.68.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.69.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.7.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.70.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.71.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.72.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.73.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.74.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.75.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.76.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.77.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.78.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.79.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.8.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.80.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.81.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.82.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.83.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.84.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.85.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.86.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.87.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.88.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.89.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.9.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.90.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.91.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.92.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.93.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.94.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.experts.95.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.gate.e_score_correction_bias": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.gate.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.mlp.shared_experts.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.post_attention_layernorm.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.k_norm.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.k_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.k_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.k_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.o_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.o_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.q_norm.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.q_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.q_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.q_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.v_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.v_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.58.self_attn.v_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.input_layernorm.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.0.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.0.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.0.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.0.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.0.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.0.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.1.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.10.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.11.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.12.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.13.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.14.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.15.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.16.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.17.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.18.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.19.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.2.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.20.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.21.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.22.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.23.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.24.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.25.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.26.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.27.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.28.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.29.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.3.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.30.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.31.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.32.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.33.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.34.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.35.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.36.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.37.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.38.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.39.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.4.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.40.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.41.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.42.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.43.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.44.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.45.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.46.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.47.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.48.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.49.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.5.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.50.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.51.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.51.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.51.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.51.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.51.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.51.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.52.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.52.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.52.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.52.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.52.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.52.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.53.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.54.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.55.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.56.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.57.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.58.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.59.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.6.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.6.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.6.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.6.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.6.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.6.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.60.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.60.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.60.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.60.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.60.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.60.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.61.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.62.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.63.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.64.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.65.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.66.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.67.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.68.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.69.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.7.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.7.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.7.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.7.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.7.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.7.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.70.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.70.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.70.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.70.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.70.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.70.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.71.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.72.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.73.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.74.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.75.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.76.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.77.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.78.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.79.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.8.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.8.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.8.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.8.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.8.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.8.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.80.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.80.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.80.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.80.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.80.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.80.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.81.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.82.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.83.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.84.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.85.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.86.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.87.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.88.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.89.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.9.down_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.9.down_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.9.gate_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.9.gate_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.9.up_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.9.up_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.mlp.experts.90.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.90.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.90.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.90.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.90.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.90.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.91.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.92.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.93.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.94.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.experts.95.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.gate.e_score_correction_bias": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.gate.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.59.mlp.shared_experts.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.59.post_attention_layernorm.weight": "model-00029-of-00045.safetensors", + "model.layers.59.self_attn.k_norm.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.k_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.k_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.k_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.o_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.o_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.q_norm.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.q_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.q_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.q_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.v_proj.bias": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.v_proj.weight": "model-00028-of-00045.safetensors", + "model.layers.59.self_attn.v_proj.weight_scale": "model-00028-of-00045.safetensors", + "model.layers.6.input_layernorm.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.0.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.0.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.0.gate_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.6.mlp.experts.0.gate_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.6.mlp.experts.0.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.0.up_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.6.mlp.experts.1.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.1.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.1.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.1.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.1.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.1.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.10.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.11.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.12.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.13.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.14.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.15.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.16.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.17.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.18.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.19.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.2.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.20.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.21.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.22.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.23.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.24.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.25.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.26.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.27.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.28.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.29.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.3.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.30.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.31.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.32.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.33.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.34.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.35.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.36.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.37.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.38.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.39.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.4.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.40.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.41.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.42.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.43.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.44.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.45.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.46.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.47.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.48.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.49.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.5.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.50.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.51.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.52.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.53.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.54.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.55.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.56.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.57.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.58.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.59.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.6.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.60.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.61.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.62.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.63.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.64.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.65.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.66.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.67.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.68.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.69.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.7.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.70.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.71.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.72.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.73.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.74.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.75.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.76.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.77.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.78.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.79.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.8.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.80.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.81.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.82.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.83.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.84.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.85.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.86.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.87.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.88.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.89.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.9.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.90.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.91.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.92.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.93.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.94.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.experts.95.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.gate.e_score_correction_bias": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.gate.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.6.mlp.shared_experts.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.6.post_attention_layernorm.weight": "model-00003-of-00045.safetensors", + "model.layers.6.self_attn.k_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.k_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.k_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.o_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.q_norm.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.q_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.q_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.v_proj.bias": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00045.safetensors", + "model.layers.6.self_attn.v_proj.weight_scale": "model-00002-of-00045.safetensors", + "model.layers.60.input_layernorm.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.0.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.1.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.10.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.11.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.12.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.13.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.14.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.15.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.16.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.17.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.18.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.19.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.2.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.20.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.21.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.22.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.23.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.24.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.25.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.26.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.27.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.28.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.29.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.3.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.30.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.31.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.32.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.33.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.34.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.35.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.36.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.37.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.38.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.39.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.4.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.40.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.41.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.42.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.43.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.44.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.45.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.46.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.47.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.48.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.49.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.5.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.50.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.51.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.52.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.53.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.54.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.55.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.56.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.57.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.58.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.59.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.6.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.60.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.61.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.62.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.63.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.64.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.65.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.66.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.67.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.68.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.69.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.7.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.70.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.71.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.72.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.73.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.74.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.75.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.76.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.77.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.78.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.79.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.8.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.80.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.81.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.82.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.83.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.84.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.85.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.86.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.87.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.88.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.89.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.9.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.90.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.91.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.92.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.93.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.94.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.experts.95.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.gate.e_score_correction_bias": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.gate.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.mlp.shared_experts.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.post_attention_layernorm.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.k_norm.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.k_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.k_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.k_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.o_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.o_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.q_norm.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.q_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.q_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.q_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.v_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.v_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.60.self_attn.v_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.input_layernorm.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.0.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.0.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.0.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.0.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.0.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.0.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.1.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.10.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.11.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.12.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.13.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.14.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.15.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.16.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.17.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.18.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.19.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.2.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.20.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.21.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.22.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.23.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.24.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.25.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.26.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.27.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.28.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.29.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.3.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.30.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.31.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.32.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.33.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.34.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.35.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.36.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.37.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.38.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.39.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.4.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.40.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.41.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.42.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.43.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.44.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.45.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.46.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.47.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.48.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.49.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.5.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.50.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.51.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.52.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.53.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.54.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.55.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.56.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.57.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.57.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.57.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.57.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.57.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.57.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.58.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.58.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.58.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.58.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.58.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.58.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.59.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.6.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.6.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.6.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.6.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.6.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.6.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.60.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.60.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.60.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.60.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.60.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.60.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.61.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.62.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.63.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.64.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.65.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.66.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.67.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.68.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.69.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.7.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.7.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.7.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.7.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.7.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.7.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.70.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.70.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.70.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.70.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.70.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.70.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.71.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.72.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.73.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.74.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.75.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.76.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.77.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.78.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.79.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.8.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.8.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.8.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.8.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.8.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.8.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.80.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.80.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.80.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.80.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.80.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.80.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.81.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.82.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.83.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.84.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.85.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.86.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.87.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.88.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.89.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.9.down_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.9.down_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.9.gate_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.9.gate_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.9.up_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.9.up_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.mlp.experts.90.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.90.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.90.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.90.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.90.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.90.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.91.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.92.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.93.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.94.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.experts.95.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.gate.e_score_correction_bias": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.gate.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.61.mlp.shared_experts.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.61.post_attention_layernorm.weight": "model-00030-of-00045.safetensors", + "model.layers.61.self_attn.k_norm.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.k_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.k_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.k_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.o_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.o_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.q_norm.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.q_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.q_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.q_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.v_proj.bias": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.v_proj.weight": "model-00029-of-00045.safetensors", + "model.layers.61.self_attn.v_proj.weight_scale": "model-00029-of-00045.safetensors", + "model.layers.62.input_layernorm.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.0.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.1.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.10.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.11.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.12.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.13.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.14.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.15.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.16.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.17.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.18.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.19.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.2.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.20.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.21.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.22.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.23.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.24.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.25.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.26.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.27.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.28.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.29.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.3.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.30.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.31.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.32.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.33.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.34.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.35.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.36.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.37.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.38.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.39.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.4.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.40.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.41.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.42.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.43.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.44.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.45.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.46.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.47.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.48.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.49.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.5.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.50.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.51.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.52.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.53.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.54.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.55.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.56.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.57.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.58.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.59.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.6.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.60.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.61.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.62.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.63.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.64.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.65.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.66.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.67.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.68.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.69.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.7.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.70.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.71.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.72.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.73.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.74.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.75.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.76.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.77.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.78.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.79.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.8.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.80.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.81.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.82.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.83.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.84.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.85.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.86.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.87.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.88.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.89.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.9.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.90.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.91.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.92.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.93.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.94.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.experts.95.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.gate.e_score_correction_bias": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.gate.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.mlp.shared_experts.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.post_attention_layernorm.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.k_norm.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.k_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.k_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.k_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.o_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.o_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.q_norm.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.q_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.q_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.q_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.v_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.v_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.62.self_attn.v_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.input_layernorm.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.0.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.0.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.0.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.0.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.0.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.0.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.1.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.10.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.11.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.12.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.13.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.14.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.15.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.16.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.17.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.18.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.19.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.2.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.20.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.21.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.22.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.23.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.24.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.25.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.26.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.27.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.28.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.29.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.3.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.30.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.31.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.32.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.33.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.34.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.35.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.36.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.37.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.38.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.39.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.4.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.40.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.41.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.42.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.43.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.44.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.45.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.46.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.47.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.48.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.49.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.5.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.50.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.51.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.52.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.53.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.54.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.55.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.56.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.57.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.58.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.59.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.6.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.60.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.61.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.62.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.63.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.63.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.63.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.63.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.63.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.63.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.64.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.64.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.64.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.64.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.64.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.64.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.65.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.66.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.67.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.68.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.69.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.7.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.7.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.7.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.7.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.7.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.7.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.70.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.70.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.70.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.70.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.70.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.70.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.71.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.72.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.73.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.74.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.75.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.76.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.77.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.78.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.79.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.8.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.8.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.8.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.8.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.8.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.8.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.80.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.80.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.80.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.80.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.80.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.80.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.81.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.82.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.83.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.84.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.85.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.86.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.87.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.88.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.89.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.9.down_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.9.down_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.9.gate_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.9.gate_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.9.up_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.9.up_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.mlp.experts.90.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.90.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.90.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.90.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.90.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.90.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.91.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.92.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.93.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.94.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.experts.95.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.gate.e_score_correction_bias": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.gate.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.63.mlp.shared_experts.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.63.post_attention_layernorm.weight": "model-00031-of-00045.safetensors", + "model.layers.63.self_attn.k_norm.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.k_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.k_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.k_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.o_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.o_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.q_norm.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.q_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.q_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.q_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.v_proj.bias": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.v_proj.weight": "model-00030-of-00045.safetensors", + "model.layers.63.self_attn.v_proj.weight_scale": "model-00030-of-00045.safetensors", + "model.layers.64.input_layernorm.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.0.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.1.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.10.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.11.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.12.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.13.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.14.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.15.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.16.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.17.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.18.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.19.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.2.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.20.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.21.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.22.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.23.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.24.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.25.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.26.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.27.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.28.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.29.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.3.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.30.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.31.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.32.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.33.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.34.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.35.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.36.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.37.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.38.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.39.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.4.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.40.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.41.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.42.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.43.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.44.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.45.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.46.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.47.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.48.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.49.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.5.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.50.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.51.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.52.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.53.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.54.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.55.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.56.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.57.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.58.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.59.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.6.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.60.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.61.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.62.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.63.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.64.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.65.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.66.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.67.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.68.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.69.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.7.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.70.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.71.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.72.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.73.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.74.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.75.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.76.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.77.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.78.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.79.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.8.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.80.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.81.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.82.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.83.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.84.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.85.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.86.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.87.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.88.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.89.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.9.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.90.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.91.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.92.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.93.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.94.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.experts.95.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.gate.e_score_correction_bias": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.gate.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.mlp.shared_experts.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.post_attention_layernorm.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.k_norm.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.k_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.k_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.k_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.o_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.o_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.q_norm.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.q_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.q_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.q_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.v_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.v_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.64.self_attn.v_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.input_layernorm.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.0.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.0.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.0.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.0.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.0.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.0.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.1.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.10.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.11.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.12.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.13.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.14.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.15.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.16.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.17.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.18.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.19.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.2.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.20.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.21.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.22.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.23.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.24.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.25.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.26.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.27.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.28.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.29.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.3.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.30.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.31.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.32.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.33.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.34.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.35.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.36.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.37.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.38.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.39.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.4.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.40.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.41.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.42.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.43.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.44.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.45.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.46.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.47.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.48.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.49.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.5.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.50.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.51.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.52.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.53.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.54.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.55.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.56.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.57.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.58.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.59.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.6.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.60.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.61.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.62.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.63.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.64.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.65.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.66.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.67.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.68.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.69.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.69.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.69.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.69.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.69.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.69.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.7.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.70.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.70.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.70.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.70.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.70.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.70.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.71.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.72.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.73.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.74.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.75.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.76.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.77.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.78.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.79.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.8.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.8.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.8.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.8.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.8.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.8.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.80.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.80.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.80.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.80.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.80.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.80.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.81.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.82.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.83.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.84.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.85.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.86.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.87.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.88.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.89.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.9.down_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.9.down_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.9.gate_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.9.gate_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.9.up_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.9.up_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.mlp.experts.90.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.90.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.90.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.90.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.90.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.90.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.91.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.92.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.93.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.94.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.experts.95.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.gate.e_score_correction_bias": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.gate.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.65.mlp.shared_experts.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.65.post_attention_layernorm.weight": "model-00032-of-00045.safetensors", + "model.layers.65.self_attn.k_norm.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.k_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.k_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.k_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.o_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.o_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.q_norm.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.q_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.q_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.q_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.v_proj.bias": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.v_proj.weight": "model-00031-of-00045.safetensors", + "model.layers.65.self_attn.v_proj.weight_scale": "model-00031-of-00045.safetensors", + "model.layers.66.input_layernorm.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.0.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.1.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.10.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.11.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.12.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.13.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.14.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.15.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.16.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.17.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.18.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.19.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.2.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.20.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.21.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.22.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.23.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.24.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.25.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.26.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.27.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.28.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.29.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.3.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.30.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.31.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.32.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.33.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.34.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.35.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.36.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.37.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.38.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.39.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.4.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.40.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.41.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.42.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.43.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.44.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.45.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.46.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.47.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.48.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.49.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.5.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.50.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.51.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.52.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.53.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.54.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.55.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.56.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.57.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.58.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.59.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.6.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.60.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.61.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.62.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.63.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.64.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.65.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.66.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.67.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.68.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.69.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.7.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.70.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.71.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.72.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.73.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.74.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.75.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.76.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.77.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.78.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.79.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.8.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.80.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.81.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.82.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.83.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.84.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.85.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.86.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.87.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.88.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.89.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.9.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.90.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.91.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.92.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.93.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.94.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.experts.95.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.gate.e_score_correction_bias": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.gate.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.mlp.shared_experts.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.post_attention_layernorm.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.k_norm.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.k_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.k_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.k_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.o_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.o_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.q_norm.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.q_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.q_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.q_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.v_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.v_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.66.self_attn.v_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.input_layernorm.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.0.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.0.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.0.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.0.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.0.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.0.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.1.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.10.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.11.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.12.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.13.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.14.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.15.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.16.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.17.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.18.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.19.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.2.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.20.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.21.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.22.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.23.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.24.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.25.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.26.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.27.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.28.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.29.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.3.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.30.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.31.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.32.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.33.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.34.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.35.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.36.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.37.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.38.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.39.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.4.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.40.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.41.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.42.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.43.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.44.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.45.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.46.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.47.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.48.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.49.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.5.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.50.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.51.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.52.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.53.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.54.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.55.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.56.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.57.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.58.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.59.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.6.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.60.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.61.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.62.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.63.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.64.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.65.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.66.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.67.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.68.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.69.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.7.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.70.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.71.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.72.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.73.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.74.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.75.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.75.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.75.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.75.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.75.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.75.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.76.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.76.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.76.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.76.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.76.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.76.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.77.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.78.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.79.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.8.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.8.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.8.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.8.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.8.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.8.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.80.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.80.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.80.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.80.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.80.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.80.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.81.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.82.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.83.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.84.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.85.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.86.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.87.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.88.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.89.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.9.down_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.9.down_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.9.gate_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.9.gate_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.9.up_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.9.up_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.mlp.experts.90.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.90.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.90.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.90.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.90.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.90.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.91.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.92.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.93.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.94.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.experts.95.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.gate.e_score_correction_bias": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.gate.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.67.mlp.shared_experts.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.67.post_attention_layernorm.weight": "model-00033-of-00045.safetensors", + "model.layers.67.self_attn.k_norm.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.k_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.k_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.k_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.o_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.o_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.q_norm.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.q_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.q_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.q_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.v_proj.bias": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.v_proj.weight": "model-00032-of-00045.safetensors", + "model.layers.67.self_attn.v_proj.weight_scale": "model-00032-of-00045.safetensors", + "model.layers.68.input_layernorm.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.0.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.1.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.10.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.11.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.12.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.13.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.14.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.15.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.16.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.17.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.18.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.19.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.2.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.20.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.21.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.22.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.23.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.24.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.25.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.26.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.27.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.28.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.29.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.3.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.30.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.31.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.32.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.33.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.34.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.35.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.36.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.37.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.38.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.39.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.4.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.40.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.41.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.42.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.43.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.44.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.45.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.46.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.47.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.48.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.49.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.5.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.50.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.51.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.52.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.53.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.54.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.55.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.56.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.57.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.58.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.59.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.6.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.60.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.61.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.62.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.63.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.64.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.65.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.66.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.67.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.68.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.69.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.7.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.70.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.71.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.72.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.73.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.74.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.75.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.76.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.77.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.78.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.79.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.8.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.80.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.81.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.82.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.83.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.84.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.85.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.86.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.87.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.88.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.89.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.9.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.90.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.91.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.92.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.93.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.94.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.experts.95.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.gate.e_score_correction_bias": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.gate.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.mlp.shared_experts.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.post_attention_layernorm.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.k_norm.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.k_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.k_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.k_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.o_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.o_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.q_norm.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.q_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.q_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.q_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.v_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.v_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.68.self_attn.v_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.input_layernorm.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.0.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.0.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.0.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.0.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.0.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.0.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.1.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.10.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.11.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.12.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.13.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.14.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.15.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.16.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.17.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.18.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.19.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.2.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.20.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.21.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.22.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.23.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.24.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.25.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.26.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.27.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.28.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.29.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.3.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.30.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.31.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.32.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.33.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.34.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.35.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.36.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.37.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.38.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.39.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.4.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.40.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.41.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.42.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.43.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.44.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.45.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.46.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.47.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.48.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.49.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.5.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.50.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.51.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.52.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.53.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.54.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.55.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.56.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.57.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.58.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.59.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.6.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.60.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.61.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.62.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.63.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.64.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.65.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.66.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.67.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.68.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.69.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.7.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.70.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.71.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.72.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.73.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.74.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.75.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.76.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.77.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.78.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.79.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.8.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.80.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.81.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.81.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.81.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.81.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.81.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.81.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.82.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.82.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.82.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.82.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.82.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.82.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.83.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.84.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.85.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.86.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.87.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.88.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.89.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.9.down_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.9.down_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.9.gate_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.9.gate_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.9.up_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.9.up_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.mlp.experts.90.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.90.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.90.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.90.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.90.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.90.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.91.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.92.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.93.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.94.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.experts.95.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.gate.e_score_correction_bias": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.gate.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.69.mlp.shared_experts.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.69.post_attention_layernorm.weight": "model-00034-of-00045.safetensors", + "model.layers.69.self_attn.k_norm.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.k_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.k_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.k_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.o_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.o_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.q_norm.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.q_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.q_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.q_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.v_proj.bias": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.v_proj.weight": "model-00033-of-00045.safetensors", + "model.layers.69.self_attn.v_proj.weight_scale": "model-00033-of-00045.safetensors", + "model.layers.7.input_layernorm.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.0.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.1.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.10.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.11.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.12.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.13.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.14.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.15.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.16.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.17.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.18.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.19.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.2.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.20.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.21.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.22.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.23.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.24.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.25.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.26.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.27.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.28.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.29.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.3.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.30.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.31.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.32.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.33.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.34.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.35.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.36.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.37.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.38.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.39.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.4.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.40.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.41.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.42.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.43.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.44.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.45.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.46.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.47.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.48.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.49.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.5.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.50.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.51.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.52.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.53.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.54.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.55.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.56.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.57.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.58.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.59.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.6.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.60.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.61.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.62.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.63.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.64.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.65.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.66.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.67.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.68.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.69.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.7.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.70.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.71.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.72.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.73.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.74.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.75.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.76.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.77.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.78.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.79.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.8.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.80.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.81.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.82.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.83.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.84.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.85.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.86.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.87.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.88.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.89.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.9.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.90.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.91.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.92.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.93.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.94.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.experts.95.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.gate.e_score_correction_bias": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.gate.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.mlp.shared_experts.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.k_norm.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.k_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.k_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.k_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.o_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.o_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.q_norm.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.q_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.q_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.q_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.v_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.v_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.7.self_attn.v_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.70.input_layernorm.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.0.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.1.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.10.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.11.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.12.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.13.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.14.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.15.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.16.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.17.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.18.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.19.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.2.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.20.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.21.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.22.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.23.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.24.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.25.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.26.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.27.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.28.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.29.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.3.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.30.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.31.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.32.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.33.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.34.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.35.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.36.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.37.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.38.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.39.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.4.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.40.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.41.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.42.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.43.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.44.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.45.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.46.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.47.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.48.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.49.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.5.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.50.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.51.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.52.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.53.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.54.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.55.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.56.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.57.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.58.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.59.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.6.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.60.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.61.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.62.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.63.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.64.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.65.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.66.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.67.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.68.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.69.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.7.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.70.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.71.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.72.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.73.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.74.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.75.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.76.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.77.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.78.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.79.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.8.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.80.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.81.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.82.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.83.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.84.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.85.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.86.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.87.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.88.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.89.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.9.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.90.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.91.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.92.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.93.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.94.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.experts.95.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.gate.e_score_correction_bias": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.gate.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.mlp.shared_experts.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.post_attention_layernorm.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.k_norm.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.k_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.k_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.k_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.o_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.o_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.q_norm.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.q_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.q_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.q_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.v_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.v_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.70.self_attn.v_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.input_layernorm.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.0.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.0.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.0.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.0.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.0.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.0.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.1.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.10.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.11.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.12.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.13.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.14.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.15.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.16.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.17.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.18.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.19.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.2.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.20.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.21.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.22.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.23.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.24.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.25.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.26.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.27.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.28.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.29.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.3.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.30.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.31.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.32.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.33.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.34.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.35.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.36.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.37.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.38.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.39.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.4.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.40.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.41.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.42.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.43.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.44.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.45.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.46.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.47.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.48.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.49.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.5.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.50.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.51.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.52.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.53.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.54.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.55.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.56.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.57.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.58.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.59.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.6.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.60.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.61.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.62.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.63.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.64.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.65.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.66.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.67.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.68.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.69.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.7.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.70.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.71.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.72.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.73.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.74.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.75.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.76.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.77.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.78.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.79.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.8.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.80.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.81.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.82.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.83.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.84.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.85.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.86.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.87.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.87.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.87.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.87.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.87.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.87.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.88.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.88.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.88.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.88.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.88.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.88.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.89.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.9.down_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.9.down_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.9.gate_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.9.gate_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.9.up_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.9.up_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.mlp.experts.90.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.90.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.90.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.90.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.90.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.90.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.91.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.92.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.93.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.94.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.experts.95.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.gate.e_score_correction_bias": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.gate.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.71.mlp.shared_experts.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.71.post_attention_layernorm.weight": "model-00035-of-00045.safetensors", + "model.layers.71.self_attn.k_norm.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.k_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.k_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.k_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.o_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.o_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.q_norm.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.q_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.q_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.q_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.v_proj.bias": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.v_proj.weight": "model-00034-of-00045.safetensors", + "model.layers.71.self_attn.v_proj.weight_scale": "model-00034-of-00045.safetensors", + "model.layers.72.input_layernorm.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.0.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.1.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.10.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.11.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.12.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.13.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.14.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.15.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.16.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.17.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.18.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.19.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.2.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.20.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.21.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.22.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.23.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.24.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.25.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.26.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.27.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.28.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.29.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.3.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.30.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.31.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.32.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.33.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.34.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.35.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.36.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.37.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.38.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.39.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.4.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.40.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.41.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.42.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.43.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.44.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.45.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.46.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.47.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.48.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.49.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.5.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.50.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.51.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.52.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.53.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.54.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.55.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.56.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.57.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.58.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.59.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.6.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.60.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.61.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.62.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.63.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.64.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.65.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.66.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.67.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.68.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.69.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.7.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.70.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.71.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.72.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.73.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.74.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.75.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.76.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.77.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.78.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.79.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.8.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.80.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.81.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.82.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.83.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.84.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.85.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.86.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.87.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.88.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.89.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.9.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.90.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.91.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.92.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.93.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.94.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.experts.95.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.gate.e_score_correction_bias": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.gate.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.mlp.shared_experts.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.post_attention_layernorm.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.k_norm.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.k_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.k_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.k_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.o_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.o_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.q_norm.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.q_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.q_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.q_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.v_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.v_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.72.self_attn.v_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.input_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.0.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.0.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.0.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.0.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.0.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.0.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.1.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.10.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.11.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.12.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.13.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.14.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.15.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.16.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.17.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.18.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.19.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.2.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.20.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.21.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.22.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.23.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.24.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.25.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.26.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.27.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.28.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.29.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.3.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.30.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.31.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.32.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.33.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.34.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.35.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.36.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.37.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.38.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.39.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.4.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.40.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.41.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.42.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.43.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.44.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.45.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.46.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.47.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.48.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.49.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.5.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.50.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.51.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.52.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.53.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.54.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.55.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.56.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.57.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.58.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.59.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.6.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.60.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.61.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.62.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.63.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.64.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.65.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.66.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.67.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.68.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.69.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.7.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.70.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.71.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.72.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.73.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.74.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.75.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.76.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.77.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.78.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.79.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.8.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.80.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.81.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.82.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.83.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.84.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.85.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.86.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.87.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.88.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.89.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.9.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.90.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.91.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.down_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.down_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.up_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.92.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.93.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.93.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.93.gate_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.93.gate_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.93.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.93.up_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.mlp.experts.94.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.94.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.94.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.94.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.94.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.94.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.experts.95.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.gate.e_score_correction_bias": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.gate.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.73.mlp.shared_experts.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.73.post_attention_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.73.self_attn.k_norm.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.k_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.k_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.k_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.o_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.o_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.q_norm.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.q_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.q_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.q_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.v_proj.bias": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.v_proj.weight": "model-00035-of-00045.safetensors", + "model.layers.73.self_attn.v_proj.weight_scale": "model-00035-of-00045.safetensors", + "model.layers.74.input_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.0.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.1.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.10.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.11.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.12.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.13.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.14.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.15.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.16.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.17.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.18.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.19.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.2.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.20.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.21.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.22.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.23.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.24.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.25.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.26.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.27.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.28.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.29.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.3.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.30.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.31.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.32.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.33.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.34.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.35.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.36.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.37.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.38.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.39.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.4.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.40.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.41.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.42.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.43.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.44.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.45.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.46.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.47.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.48.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.49.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.5.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.50.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.51.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.52.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.53.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.54.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.55.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.56.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.57.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.58.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.59.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.6.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.60.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.61.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.62.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.63.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.64.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.65.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.66.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.67.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.68.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.69.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.7.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.70.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.71.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.72.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.73.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.74.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.75.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.76.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.77.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.78.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.79.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.8.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.80.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.81.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.82.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.83.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.84.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.85.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.86.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.87.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.88.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.89.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.9.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.90.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.91.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.92.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.93.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.94.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.experts.95.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.gate.e_score_correction_bias": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.gate.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.mlp.shared_experts.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.post_attention_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.k_norm.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.k_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.k_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.k_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.o_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.o_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.q_norm.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.q_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.q_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.q_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.v_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.v_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.74.self_attn.v_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.input_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.0.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.1.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.10.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.11.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.12.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.13.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.14.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.15.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.16.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.17.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.18.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.19.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.2.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.20.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.21.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.22.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.23.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.24.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.25.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.26.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.27.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.28.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.29.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.3.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.30.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.31.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.32.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.33.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.34.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.35.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.36.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.37.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.38.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.39.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.4.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.40.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.41.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.42.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.43.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.44.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.45.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.46.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.47.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.48.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.49.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.5.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.50.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.51.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.52.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.53.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.54.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.55.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.56.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.57.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.58.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.59.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.6.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.60.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.61.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.62.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.63.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.64.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.65.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.66.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.67.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.68.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.69.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.7.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.70.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.71.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.72.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.73.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.74.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.75.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.76.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.77.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.78.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.79.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.8.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.80.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.81.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.82.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.83.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.84.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.85.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.86.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.87.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.88.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.89.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.9.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.90.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.91.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.92.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.93.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.94.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.experts.95.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.gate.e_score_correction_bias": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.gate.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.down_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.down_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.gate_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.gate_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.up_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.mlp.shared_experts.up_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.post_attention_layernorm.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.k_norm.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.k_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.k_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.k_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.o_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.o_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.q_norm.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.q_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.q_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.q_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.v_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.v_proj.weight": "model-00036-of-00045.safetensors", + "model.layers.75.self_attn.v_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.76.input_layernorm.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.0.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.1.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.10.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.11.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.12.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.13.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.14.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.15.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.16.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.17.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.18.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.19.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.2.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.20.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.21.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.22.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.23.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.24.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.25.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.26.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.27.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.28.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.29.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.3.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.30.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.31.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.32.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.33.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.34.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.35.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.36.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.37.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.38.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.39.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.4.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.40.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.41.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.42.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.43.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.44.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.45.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.46.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.47.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.48.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.49.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.5.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.50.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.51.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.52.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.53.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.54.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.55.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.56.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.57.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.58.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.59.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.6.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.60.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.61.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.62.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.63.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.64.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.65.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.66.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.67.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.68.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.69.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.7.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.70.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.71.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.72.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.73.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.74.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.75.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.76.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.77.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.78.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.79.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.8.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.80.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.81.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.82.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.83.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.84.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.85.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.86.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.87.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.88.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.89.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.9.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.90.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.91.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.92.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.93.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.94.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.experts.95.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.gate.e_score_correction_bias": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.gate.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.mlp.shared_experts.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.post_attention_layernorm.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.k_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.k_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.k_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.k_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.o_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.o_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.q_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.q_proj.bias": "model-00036-of-00045.safetensors", + "model.layers.76.self_attn.q_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.q_proj.weight_scale": "model-00036-of-00045.safetensors", + "model.layers.76.self_attn.v_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.v_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.76.self_attn.v_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.input_layernorm.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.0.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.1.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.10.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.11.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.12.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.13.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.14.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.15.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.16.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.17.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.18.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.19.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.2.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.20.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.21.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.22.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.23.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.24.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.25.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.26.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.27.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.28.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.29.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.3.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.30.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.31.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.32.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.33.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.34.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.35.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.36.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.37.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.38.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.39.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.4.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.40.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.41.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.42.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.43.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.44.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.45.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.46.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.47.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.48.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.49.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.5.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.50.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.51.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.52.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.53.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.54.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.55.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.56.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.57.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.58.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.59.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.6.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.60.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.61.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.62.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.63.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.64.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.65.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.66.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.67.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.68.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.69.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.7.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.70.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.71.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.72.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.73.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.74.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.75.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.76.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.77.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.78.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.79.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.8.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.80.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.81.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.82.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.83.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.84.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.85.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.86.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.87.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.88.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.89.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.9.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.90.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.91.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.92.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.93.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.94.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.experts.95.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.gate.e_score_correction_bias": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.gate.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.down_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.down_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.up_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.mlp.shared_experts.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.post_attention_layernorm.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.k_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.k_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.k_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.k_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.o_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.o_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.q_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.q_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.q_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.q_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.v_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.v_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.77.self_attn.v_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.input_layernorm.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.0.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.0.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.0.gate_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.78.mlp.experts.0.gate_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.mlp.experts.0.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.0.up_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.mlp.experts.1.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.1.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.1.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.1.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.1.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.1.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.10.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.11.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.12.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.13.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.14.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.15.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.16.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.17.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.18.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.19.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.2.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.20.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.21.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.22.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.23.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.24.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.25.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.26.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.27.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.28.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.29.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.3.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.30.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.31.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.32.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.33.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.34.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.35.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.36.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.37.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.38.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.39.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.4.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.40.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.41.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.42.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.43.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.44.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.45.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.46.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.47.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.48.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.49.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.5.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.50.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.51.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.52.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.53.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.54.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.55.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.56.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.57.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.58.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.59.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.6.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.60.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.61.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.62.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.63.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.64.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.65.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.66.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.67.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.68.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.69.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.7.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.70.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.71.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.72.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.73.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.74.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.75.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.76.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.77.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.78.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.79.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.8.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.80.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.81.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.82.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.83.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.84.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.85.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.86.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.87.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.88.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.89.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.9.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.90.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.91.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.92.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.93.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.94.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.experts.95.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.gate.e_score_correction_bias": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.gate.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.78.mlp.shared_experts.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.78.post_attention_layernorm.weight": "model-00038-of-00045.safetensors", + "model.layers.78.self_attn.k_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.k_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.k_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.k_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.o_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.o_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.q_norm.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.q_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.q_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.q_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.v_proj.bias": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.v_proj.weight": "model-00037-of-00045.safetensors", + "model.layers.78.self_attn.v_proj.weight_scale": "model-00037-of-00045.safetensors", + "model.layers.79.input_layernorm.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.0.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.1.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.10.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.11.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.12.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.13.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.14.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.15.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.16.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.17.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.18.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.19.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.2.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.20.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.21.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.22.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.23.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.24.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.25.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.26.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.27.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.28.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.29.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.3.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.30.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.31.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.32.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.33.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.34.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.35.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.36.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.37.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.38.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.39.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.4.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.40.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.41.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.42.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.43.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.44.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.45.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.46.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.47.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.48.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.49.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.5.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.50.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.51.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.52.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.53.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.54.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.55.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.56.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.57.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.58.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.59.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.6.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.60.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.61.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.62.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.63.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.64.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.65.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.66.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.67.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.68.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.69.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.7.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.70.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.71.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.72.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.73.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.74.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.75.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.76.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.77.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.78.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.79.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.8.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.80.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.81.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.82.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.83.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.84.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.85.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.86.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.87.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.88.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.89.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.9.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.90.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.91.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.92.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.93.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.94.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.experts.95.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.gate.e_score_correction_bias": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.gate.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.mlp.shared_experts.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.post_attention_layernorm.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.k_norm.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.k_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.k_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.k_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.o_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.o_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.q_norm.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.q_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.q_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.q_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.v_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.v_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.79.self_attn.v_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.8.input_layernorm.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.0.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.0.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.0.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.0.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.0.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.0.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.1.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.10.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.10.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.10.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.10.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.10.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.10.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.11.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.12.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.13.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.14.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.15.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.16.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.17.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.18.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.19.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.2.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.2.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.2.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.2.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.2.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.2.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.20.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.20.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.20.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.20.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.20.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.20.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.21.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.22.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.23.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.24.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.25.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.26.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.27.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.28.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.29.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.3.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.3.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.3.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.3.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.3.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.3.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.30.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.30.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.30.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.30.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.30.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.30.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.31.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.32.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.33.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.34.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.35.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.36.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.37.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.38.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.39.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.4.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.4.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.4.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.4.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.4.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.4.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.40.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.40.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.40.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.40.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.40.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.40.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.41.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.42.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.43.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.44.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.45.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.46.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.47.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.48.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.49.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.5.down_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.5.down_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.5.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.5.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.5.up_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.5.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.50.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.50.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.50.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.50.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.50.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.50.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.51.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.52.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.53.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.54.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.55.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.56.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.57.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.58.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.59.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.6.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.6.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.6.gate_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.6.gate_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.6.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.6.up_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.mlp.experts.60.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.60.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.60.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.60.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.60.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.60.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.61.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.62.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.63.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.64.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.65.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.66.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.67.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.68.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.69.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.7.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.70.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.71.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.72.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.73.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.74.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.75.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.76.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.77.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.78.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.79.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.8.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.80.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.81.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.82.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.83.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.84.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.85.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.86.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.87.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.88.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.89.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.9.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.90.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.91.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.92.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.93.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.94.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.experts.95.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.gate.e_score_correction_bias": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.gate.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.8.mlp.shared_experts.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.8.post_attention_layernorm.weight": "model-00004-of-00045.safetensors", + "model.layers.8.self_attn.k_norm.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.k_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.k_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.o_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.o_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.q_norm.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.q_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.q_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.v_proj.bias": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00045.safetensors", + "model.layers.8.self_attn.v_proj.weight_scale": "model-00003-of-00045.safetensors", + "model.layers.80.input_layernorm.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.0.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.0.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.0.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.0.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.0.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.0.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.1.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.10.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.10.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.10.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.10.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.10.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.10.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.11.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.12.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.13.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.14.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.15.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.16.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.17.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.18.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.19.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.2.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.2.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.2.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.2.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.2.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.2.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.20.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.20.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.20.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.20.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.20.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.20.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.21.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.22.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.23.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.24.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.25.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.26.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.27.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.28.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.29.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.3.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.3.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.3.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.3.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.3.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.3.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.30.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.30.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.30.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.30.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.30.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.30.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.31.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.32.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.33.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.34.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.35.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.36.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.37.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.38.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.39.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.4.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.4.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.4.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.4.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.4.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.4.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.40.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.40.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.40.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.40.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.40.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.40.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.41.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.42.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.43.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.44.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.45.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.46.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.47.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.48.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.49.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.5.down_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.5.down_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.5.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.5.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.5.up_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.5.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.50.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.50.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.50.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.50.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.50.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.50.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.51.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.52.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.53.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.54.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.55.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.56.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.57.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.58.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.59.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.6.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.6.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.6.gate_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.6.gate_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.6.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.6.up_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.mlp.experts.60.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.60.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.60.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.60.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.60.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.60.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.61.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.62.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.63.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.64.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.65.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.66.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.67.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.68.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.69.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.7.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.70.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.71.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.72.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.73.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.74.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.75.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.76.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.77.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.78.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.79.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.8.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.80.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.81.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.82.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.83.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.84.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.85.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.86.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.87.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.88.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.89.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.9.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.90.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.91.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.92.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.93.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.94.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.experts.95.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.gate.e_score_correction_bias": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.gate.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.80.mlp.shared_experts.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.80.post_attention_layernorm.weight": "model-00039-of-00045.safetensors", + "model.layers.80.self_attn.k_norm.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.k_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.k_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.k_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.o_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.o_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.q_norm.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.q_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.q_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.q_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.v_proj.bias": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.v_proj.weight": "model-00038-of-00045.safetensors", + "model.layers.80.self_attn.v_proj.weight_scale": "model-00038-of-00045.safetensors", + "model.layers.81.input_layernorm.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.0.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.1.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.10.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.11.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.12.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.13.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.14.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.15.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.16.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.17.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.18.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.19.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.2.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.20.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.21.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.22.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.23.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.24.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.25.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.26.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.27.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.28.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.29.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.3.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.30.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.31.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.32.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.33.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.34.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.35.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.36.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.37.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.38.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.39.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.4.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.40.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.41.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.42.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.43.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.44.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.45.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.46.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.47.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.48.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.49.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.5.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.50.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.51.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.52.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.53.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.54.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.55.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.56.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.57.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.58.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.59.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.6.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.60.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.61.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.62.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.63.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.64.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.65.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.66.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.67.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.68.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.69.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.7.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.70.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.71.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.72.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.73.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.74.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.75.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.76.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.77.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.78.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.79.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.8.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.80.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.81.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.82.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.83.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.84.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.85.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.86.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.87.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.88.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.89.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.9.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.90.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.91.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.92.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.93.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.94.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.experts.95.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.gate.e_score_correction_bias": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.gate.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.mlp.shared_experts.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.post_attention_layernorm.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.k_norm.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.k_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.k_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.k_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.o_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.o_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.q_norm.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.q_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.q_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.q_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.v_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.v_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.81.self_attn.v_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.input_layernorm.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.0.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.0.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.0.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.0.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.0.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.0.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.1.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.10.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.11.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.12.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.12.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.12.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.12.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.12.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.12.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.13.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.13.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.13.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.13.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.13.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.13.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.14.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.15.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.16.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.17.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.18.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.19.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.2.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.2.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.2.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.2.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.2.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.2.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.20.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.20.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.20.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.20.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.20.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.20.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.21.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.22.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.23.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.24.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.25.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.26.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.27.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.28.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.29.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.3.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.3.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.3.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.3.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.3.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.3.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.30.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.30.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.30.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.30.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.30.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.30.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.31.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.32.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.33.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.34.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.35.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.36.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.37.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.38.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.39.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.4.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.4.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.4.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.4.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.4.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.4.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.40.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.40.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.40.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.40.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.40.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.40.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.41.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.42.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.43.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.44.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.45.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.46.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.47.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.48.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.49.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.5.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.5.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.5.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.5.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.5.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.5.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.50.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.50.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.50.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.50.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.50.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.50.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.51.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.52.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.53.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.54.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.55.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.56.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.57.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.58.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.59.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.6.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.6.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.6.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.6.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.6.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.6.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.60.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.60.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.60.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.60.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.60.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.60.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.61.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.62.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.63.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.64.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.65.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.66.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.67.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.68.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.69.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.7.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.7.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.7.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.7.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.7.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.7.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.70.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.70.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.70.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.70.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.70.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.70.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.71.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.72.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.73.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.74.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.75.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.76.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.77.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.78.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.79.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.8.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.8.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.8.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.8.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.8.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.8.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.80.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.80.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.80.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.80.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.80.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.80.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.81.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.82.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.83.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.84.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.85.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.86.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.87.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.88.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.89.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.9.down_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.9.down_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.9.gate_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.9.gate_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.9.up_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.9.up_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.mlp.experts.90.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.90.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.90.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.90.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.90.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.90.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.91.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.92.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.93.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.94.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.experts.95.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.gate.e_score_correction_bias": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.gate.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.82.mlp.shared_experts.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.82.post_attention_layernorm.weight": "model-00040-of-00045.safetensors", + "model.layers.82.self_attn.k_norm.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.k_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.k_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.k_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.o_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.o_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.q_norm.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.q_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.q_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.q_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.v_proj.bias": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.v_proj.weight": "model-00039-of-00045.safetensors", + "model.layers.82.self_attn.v_proj.weight_scale": "model-00039-of-00045.safetensors", + "model.layers.83.input_layernorm.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.0.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.1.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.10.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.11.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.12.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.13.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.14.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.15.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.16.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.17.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.18.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.19.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.2.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.20.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.21.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.22.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.23.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.24.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.25.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.26.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.27.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.28.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.29.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.3.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.30.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.31.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.32.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.33.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.34.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.35.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.36.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.37.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.38.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.39.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.4.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.40.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.41.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.42.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.43.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.44.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.45.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.46.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.47.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.48.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.49.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.5.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.50.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.51.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.52.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.53.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.54.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.55.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.56.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.57.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.58.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.59.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.6.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.60.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.61.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.62.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.63.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.64.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.65.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.66.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.67.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.68.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.69.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.7.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.70.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.71.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.72.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.73.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.74.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.75.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.76.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.77.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.78.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.79.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.8.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.80.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.81.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.82.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.83.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.84.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.85.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.86.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.87.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.88.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.89.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.9.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.90.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.91.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.92.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.93.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.94.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.experts.95.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.gate.e_score_correction_bias": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.gate.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.mlp.shared_experts.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.post_attention_layernorm.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.k_norm.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.k_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.k_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.k_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.o_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.o_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.q_norm.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.q_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.q_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.q_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.v_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.v_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.83.self_attn.v_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.input_layernorm.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.0.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.0.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.0.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.0.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.0.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.0.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.1.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.10.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.11.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.12.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.13.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.14.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.15.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.16.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.17.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.18.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.18.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.18.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.18.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.18.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.18.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.19.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.19.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.19.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.19.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.19.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.19.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.2.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.2.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.2.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.2.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.2.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.2.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.20.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.20.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.20.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.20.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.20.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.20.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.21.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.22.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.23.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.24.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.25.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.26.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.27.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.28.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.29.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.3.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.3.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.3.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.3.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.3.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.3.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.30.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.30.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.30.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.30.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.30.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.30.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.31.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.32.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.33.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.34.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.35.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.36.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.37.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.38.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.39.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.4.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.4.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.4.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.4.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.4.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.4.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.40.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.40.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.40.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.40.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.40.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.40.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.41.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.42.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.43.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.44.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.45.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.46.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.47.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.48.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.49.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.5.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.5.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.5.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.5.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.5.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.5.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.50.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.50.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.50.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.50.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.50.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.50.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.51.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.52.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.53.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.54.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.55.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.56.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.57.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.58.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.59.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.6.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.6.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.6.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.6.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.6.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.6.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.60.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.60.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.60.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.60.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.60.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.60.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.61.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.62.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.63.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.64.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.65.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.66.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.67.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.68.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.69.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.7.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.7.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.7.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.7.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.7.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.7.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.70.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.70.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.70.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.70.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.70.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.70.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.71.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.72.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.73.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.74.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.75.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.76.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.77.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.78.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.79.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.8.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.8.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.8.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.8.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.8.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.8.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.80.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.80.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.80.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.80.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.80.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.80.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.81.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.82.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.83.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.84.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.85.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.86.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.87.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.88.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.89.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.9.down_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.9.down_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.9.gate_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.9.gate_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.9.up_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.9.up_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.mlp.experts.90.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.90.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.90.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.90.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.90.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.90.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.91.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.92.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.93.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.94.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.experts.95.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.gate.e_score_correction_bias": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.gate.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.84.mlp.shared_experts.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.84.post_attention_layernorm.weight": "model-00041-of-00045.safetensors", + "model.layers.84.self_attn.k_norm.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.k_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.k_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.k_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.o_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.o_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.q_norm.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.q_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.q_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.q_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.v_proj.bias": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.v_proj.weight": "model-00040-of-00045.safetensors", + "model.layers.84.self_attn.v_proj.weight_scale": "model-00040-of-00045.safetensors", + "model.layers.85.input_layernorm.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.0.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.1.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.10.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.11.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.12.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.13.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.14.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.15.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.16.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.17.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.18.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.19.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.2.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.20.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.21.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.22.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.23.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.24.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.25.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.26.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.27.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.28.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.29.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.3.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.30.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.31.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.32.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.33.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.34.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.35.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.36.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.37.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.38.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.39.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.4.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.40.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.41.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.42.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.43.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.44.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.45.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.46.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.47.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.48.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.49.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.5.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.50.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.51.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.52.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.53.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.54.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.55.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.56.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.57.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.58.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.59.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.6.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.60.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.61.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.62.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.63.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.64.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.65.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.66.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.67.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.68.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.69.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.7.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.70.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.71.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.72.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.73.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.74.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.75.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.76.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.77.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.78.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.79.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.8.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.80.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.81.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.82.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.83.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.84.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.85.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.86.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.87.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.88.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.89.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.9.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.90.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.91.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.92.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.93.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.94.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.experts.95.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.gate.e_score_correction_bias": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.gate.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.mlp.shared_experts.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.post_attention_layernorm.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.k_norm.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.k_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.k_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.k_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.o_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.o_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.q_norm.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.q_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.q_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.q_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.v_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.v_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.85.self_attn.v_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.input_layernorm.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.0.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.0.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.0.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.0.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.0.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.0.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.1.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.10.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.11.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.12.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.13.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.14.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.15.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.16.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.17.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.18.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.19.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.2.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.20.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.21.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.22.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.23.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.24.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.24.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.24.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.24.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.24.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.24.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.25.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.25.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.25.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.25.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.25.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.25.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.26.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.27.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.28.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.29.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.3.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.3.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.3.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.3.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.3.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.3.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.30.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.30.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.30.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.30.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.30.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.30.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.31.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.32.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.33.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.34.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.35.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.36.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.37.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.38.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.39.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.4.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.4.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.4.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.4.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.4.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.4.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.40.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.40.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.40.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.40.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.40.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.40.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.41.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.42.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.43.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.44.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.45.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.46.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.47.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.48.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.49.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.5.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.5.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.5.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.5.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.5.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.5.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.50.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.50.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.50.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.50.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.50.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.50.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.51.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.52.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.53.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.54.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.55.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.56.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.57.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.58.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.59.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.6.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.6.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.6.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.6.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.6.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.6.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.60.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.60.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.60.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.60.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.60.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.60.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.61.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.62.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.63.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.64.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.65.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.66.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.67.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.68.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.69.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.7.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.7.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.7.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.7.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.7.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.7.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.70.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.70.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.70.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.70.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.70.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.70.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.71.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.72.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.73.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.74.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.75.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.76.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.77.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.78.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.79.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.8.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.8.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.8.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.8.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.8.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.8.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.80.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.80.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.80.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.80.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.80.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.80.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.81.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.82.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.83.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.84.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.85.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.86.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.87.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.88.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.89.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.9.down_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.9.down_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.9.gate_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.9.gate_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.9.up_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.9.up_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.mlp.experts.90.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.90.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.90.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.90.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.90.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.90.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.91.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.92.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.93.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.94.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.experts.95.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.gate.e_score_correction_bias": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.gate.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.86.mlp.shared_experts.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.86.post_attention_layernorm.weight": "model-00042-of-00045.safetensors", + "model.layers.86.self_attn.k_norm.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.k_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.k_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.k_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.o_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.o_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.q_norm.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.q_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.q_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.q_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.v_proj.bias": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.v_proj.weight": "model-00041-of-00045.safetensors", + "model.layers.86.self_attn.v_proj.weight_scale": "model-00041-of-00045.safetensors", + "model.layers.87.input_layernorm.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.0.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.1.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.10.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.11.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.12.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.13.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.14.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.15.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.16.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.17.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.18.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.19.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.2.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.20.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.21.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.22.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.23.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.24.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.25.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.26.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.27.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.28.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.29.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.3.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.30.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.31.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.32.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.33.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.34.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.35.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.36.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.37.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.38.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.39.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.4.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.40.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.41.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.42.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.43.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.44.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.45.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.46.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.47.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.48.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.49.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.5.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.50.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.51.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.52.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.53.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.54.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.55.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.56.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.57.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.58.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.59.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.6.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.60.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.61.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.62.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.63.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.64.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.65.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.66.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.67.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.68.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.69.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.7.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.70.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.71.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.72.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.73.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.74.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.75.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.76.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.77.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.78.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.79.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.8.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.80.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.81.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.82.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.83.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.84.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.85.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.86.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.87.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.88.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.89.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.9.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.90.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.91.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.92.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.93.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.94.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.experts.95.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.gate.e_score_correction_bias": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.gate.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.mlp.shared_experts.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.post_attention_layernorm.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.k_norm.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.k_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.k_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.k_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.o_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.o_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.q_norm.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.q_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.q_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.q_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.v_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.v_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.87.self_attn.v_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.input_layernorm.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.0.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.0.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.0.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.0.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.0.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.0.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.1.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.10.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.11.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.12.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.13.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.14.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.15.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.16.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.17.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.18.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.19.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.2.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.20.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.21.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.22.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.23.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.24.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.25.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.26.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.27.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.28.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.29.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.3.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.30.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.30.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.30.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.30.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.30.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.30.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.31.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.31.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.31.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.31.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.31.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.31.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.32.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.33.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.34.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.35.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.36.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.37.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.38.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.39.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.4.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.4.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.4.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.4.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.4.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.4.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.40.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.40.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.40.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.40.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.40.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.40.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.41.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.42.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.43.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.44.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.45.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.46.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.47.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.48.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.49.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.5.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.5.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.5.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.5.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.5.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.5.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.50.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.50.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.50.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.50.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.50.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.50.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.51.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.52.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.53.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.54.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.55.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.56.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.57.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.58.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.59.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.6.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.6.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.6.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.6.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.6.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.6.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.60.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.60.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.60.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.60.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.60.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.60.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.61.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.62.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.63.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.64.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.65.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.66.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.67.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.68.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.69.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.7.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.7.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.7.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.7.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.7.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.7.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.70.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.70.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.70.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.70.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.70.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.70.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.71.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.72.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.73.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.74.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.75.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.76.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.77.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.78.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.79.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.8.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.8.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.8.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.8.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.8.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.8.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.80.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.80.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.80.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.80.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.80.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.80.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.81.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.82.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.83.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.84.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.85.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.86.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.87.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.88.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.89.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.9.down_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.9.down_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.9.gate_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.9.gate_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.9.up_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.9.up_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.mlp.experts.90.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.90.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.90.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.90.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.90.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.90.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.91.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.92.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.93.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.94.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.experts.95.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.gate.e_score_correction_bias": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.gate.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.88.mlp.shared_experts.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.88.post_attention_layernorm.weight": "model-00043-of-00045.safetensors", + "model.layers.88.self_attn.k_norm.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.k_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.k_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.k_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.o_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.o_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.q_norm.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.q_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.q_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.q_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.v_proj.bias": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.v_proj.weight": "model-00042-of-00045.safetensors", + "model.layers.88.self_attn.v_proj.weight_scale": "model-00042-of-00045.safetensors", + "model.layers.89.input_layernorm.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.0.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.1.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.10.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.11.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.12.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.13.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.14.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.15.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.16.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.17.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.18.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.19.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.2.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.20.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.21.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.22.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.23.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.24.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.25.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.26.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.27.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.28.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.29.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.3.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.30.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.31.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.32.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.33.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.34.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.35.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.36.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.37.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.38.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.39.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.4.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.40.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.41.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.42.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.43.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.44.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.45.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.46.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.47.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.48.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.49.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.5.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.50.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.51.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.52.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.53.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.54.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.55.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.56.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.57.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.58.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.59.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.6.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.60.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.61.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.62.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.63.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.64.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.65.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.66.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.67.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.68.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.69.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.7.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.70.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.71.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.72.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.73.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.74.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.75.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.76.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.77.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.78.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.79.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.8.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.80.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.81.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.82.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.83.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.84.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.85.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.86.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.87.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.88.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.89.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.9.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.90.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.91.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.92.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.93.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.94.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.experts.95.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.gate.e_score_correction_bias": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.gate.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.mlp.shared_experts.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.post_attention_layernorm.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.k_norm.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.k_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.k_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.k_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.o_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.o_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.q_norm.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.q_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.q_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.q_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.v_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.v_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.89.self_attn.v_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.9.input_layernorm.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.0.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.1.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.10.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.11.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.12.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.13.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.14.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.15.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.16.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.17.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.18.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.19.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.2.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.20.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.21.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.22.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.23.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.24.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.25.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.26.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.27.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.28.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.29.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.3.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.30.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.31.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.32.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.33.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.34.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.35.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.36.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.37.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.38.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.39.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.4.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.40.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.41.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.42.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.43.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.44.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.45.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.46.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.47.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.48.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.49.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.5.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.50.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.51.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.52.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.53.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.54.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.55.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.56.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.57.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.58.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.59.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.6.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.60.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.61.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.62.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.63.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.64.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.65.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.66.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.67.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.68.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.69.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.7.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.70.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.71.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.72.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.73.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.74.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.75.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.76.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.77.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.78.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.79.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.8.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.80.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.81.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.82.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.83.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.84.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.85.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.86.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.87.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.88.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.89.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.9.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.90.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.91.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.92.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.93.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.94.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.experts.95.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.gate.e_score_correction_bias": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.gate.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.down_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.down_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.gate_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.gate_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.up_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.mlp.shared_experts.up_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.post_attention_layernorm.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.k_norm.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.k_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.k_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.k_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.o_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.o_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.q_norm.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.q_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.q_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.q_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.v_proj.bias": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.v_proj.weight": "model-00004-of-00045.safetensors", + "model.layers.9.self_attn.v_proj.weight_scale": "model-00004-of-00045.safetensors", + "model.layers.90.input_layernorm.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.0.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.0.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.0.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.0.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.0.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.0.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.1.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.10.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.11.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.12.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.13.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.14.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.15.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.16.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.17.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.18.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.19.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.2.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.20.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.21.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.22.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.23.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.24.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.25.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.26.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.27.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.28.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.29.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.3.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.30.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.31.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.32.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.33.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.34.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.35.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.36.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.36.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.36.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.36.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.36.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.36.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.37.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.37.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.37.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.37.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.37.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.37.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.38.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.39.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.4.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.4.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.4.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.4.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.4.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.4.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.40.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.40.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.40.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.40.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.40.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.40.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.41.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.42.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.43.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.44.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.45.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.46.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.47.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.48.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.49.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.5.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.5.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.5.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.5.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.5.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.5.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.50.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.50.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.50.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.50.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.50.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.50.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.51.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.52.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.53.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.54.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.55.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.56.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.57.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.58.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.59.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.6.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.6.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.6.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.6.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.6.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.6.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.60.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.60.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.60.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.60.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.60.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.60.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.61.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.62.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.63.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.64.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.65.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.66.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.67.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.68.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.69.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.7.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.7.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.7.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.7.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.7.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.7.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.70.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.70.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.70.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.70.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.70.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.70.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.71.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.72.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.73.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.74.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.75.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.76.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.77.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.78.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.79.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.8.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.8.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.8.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.8.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.8.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.8.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.80.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.80.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.80.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.80.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.80.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.80.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.81.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.82.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.83.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.84.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.85.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.86.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.87.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.88.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.89.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.9.down_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.9.down_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.9.gate_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.9.gate_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.9.up_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.9.up_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.mlp.experts.90.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.90.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.90.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.90.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.90.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.90.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.91.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.92.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.93.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.94.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.experts.95.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.gate.e_score_correction_bias": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.gate.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.90.mlp.shared_experts.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.90.post_attention_layernorm.weight": "model-00044-of-00045.safetensors", + "model.layers.90.self_attn.k_norm.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.k_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.k_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.k_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.o_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.o_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.q_norm.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.q_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.q_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.q_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.v_proj.bias": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.v_proj.weight": "model-00043-of-00045.safetensors", + "model.layers.90.self_attn.v_proj.weight_scale": "model-00043-of-00045.safetensors", + "model.layers.91.input_layernorm.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.0.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.1.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.10.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.11.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.12.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.13.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.14.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.15.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.16.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.17.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.18.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.19.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.2.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.20.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.21.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.22.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.23.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.24.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.25.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.26.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.27.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.28.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.29.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.3.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.30.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.31.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.32.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.33.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.34.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.35.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.36.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.37.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.38.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.39.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.4.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.40.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.41.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.42.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.43.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.44.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.45.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.46.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.47.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.48.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.49.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.5.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.50.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.51.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.52.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.53.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.54.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.55.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.56.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.57.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.58.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.59.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.6.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.60.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.61.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.62.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.63.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.64.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.65.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.66.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.67.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.68.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.69.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.7.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.70.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.71.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.72.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.73.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.74.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.75.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.76.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.77.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.78.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.79.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.8.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.80.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.81.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.82.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.83.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.84.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.85.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.86.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.87.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.88.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.89.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.9.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.90.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.91.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.92.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.93.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.94.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.experts.95.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.gate.e_score_correction_bias": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.gate.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.down_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.down_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.gate_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.gate_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.up_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.mlp.shared_experts.up_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.post_attention_layernorm.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.k_norm.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.k_proj.bias": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.k_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.k_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.o_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.o_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.q_norm.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.q_proj.bias": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.q_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.q_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.v_proj.bias": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.v_proj.weight": "model-00044-of-00045.safetensors", + "model.layers.91.self_attn.v_proj.weight_scale": "model-00044-of-00045.safetensors", + "model.norm.weight": "model-00044-of-00045.safetensors" + } +} diff --git a/special_tokens_map.json b/special_tokens_map.json new file mode 100755 index 0000000000000000000000000000000000000000..9028cf84013844f17d7616bdec1d88e977924434 --- /dev/null +++ b/special_tokens_map.json @@ -0,0 +1,40 @@ +{ + "additional_special_tokens": [ + "<|endoftext|>", + "[MASK]", + "[gMASK]", + "[sMASK]", + "", + "", + "<|system|>", + "<|user|>", + "<|assistant|>", + "<|observation|>", + "<|begin_of_image|>", + "<|end_of_image|>", + "<|begin_of_video|>", + "<|end_of_video|>", + "<|begin_of_audio|>", + "<|end_of_audio|>", + "<|begin_of_transcription|>", + "<|end_of_transcription|>", + "<|code_prefix|>", + "<|code_middle|>", + "<|code_suffix|>", + "/nothink" + ], + "eos_token": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + }, + "pad_token": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false + } +} diff --git a/tokenizer.json b/tokenizer.json new file mode 100755 index 0000000000000000000000000000000000000000..e3ed3c66baf1ec4de61840b0abf02142687bfed8 --- /dev/null +++ b/tokenizer.json @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bda8e2146c3bb7b7e0fc96dcc4f0aeff041c6c27952e3ace0665663ebff346ba +size 19970700 diff --git a/tokenizer_config.json b/tokenizer_config.json new file mode 100755 index 0000000000000000000000000000000000000000..75e11cfb2e0cc09f19391ec2278b4825a4c3fae9 --- /dev/null +++ b/tokenizer_config.json @@ -0,0 +1,325 @@ +{ + "added_tokens_decoder": { + "151329": { + "content": "<|endoftext|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151330": { + "content": "[MASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151331": { + "content": "[gMASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151332": { + "content": "[sMASK]", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151333": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151334": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151335": { + "content": "<|system|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151336": { + "content": "<|user|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151337": { + "content": "<|assistant|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151338": { + "content": "<|observation|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151339": { + "content": "<|begin_of_image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151340": { + "content": "<|end_of_image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151341": { + "content": "<|begin_of_video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151342": { + "content": "<|end_of_video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151343": { + "content": "<|begin_of_audio|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151344": { + "content": "<|end_of_audio|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151345": { + "content": "<|begin_of_transcription|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151346": { + "content": "<|end_of_transcription|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151347": { + "content": "<|code_prefix|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151348": { + "content": "<|code_middle|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151349": { + "content": "<|code_suffix|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151350": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151351": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151352": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151353": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151354": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151355": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151356": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151357": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151358": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151359": { + "content": "", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151360": { + "content": "/nothink", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": true + }, + "151361": { + "content": "<|begin_of_box|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151362": { + "content": "<|end_of_box|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151363": { + "content": "<|image|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + }, + "151364": { + "content": "<|video|>", + "lstrip": false, + "normalized": false, + "rstrip": false, + "single_word": false, + "special": false + } + }, + "additional_special_tokens": [ + "<|endoftext|>", + "[MASK]", + "[gMASK]", + "[sMASK]", + "", + "", + "<|system|>", + "<|user|>", + "<|assistant|>", + "<|observation|>", + "<|begin_of_image|>", + "<|end_of_image|>", + "<|begin_of_video|>", + "<|end_of_video|>", + "<|begin_of_audio|>", + "<|end_of_audio|>", + "<|begin_of_transcription|>", + "<|end_of_transcription|>", + "<|code_prefix|>", + "<|code_middle|>", + "<|code_suffix|>", + "/nothink" + ], + "clean_up_tokenization_spaces": false, + "do_lower_case": false, + "eos_token": "<|endoftext|>", + "extra_special_tokens": {}, + "model_max_length": 128000, + "pad_token": "<|endoftext|>", + "padding_side": "left", + "remove_space": false, + "tokenizer_class": "PreTrainedTokenizerFast" +}