使用示例代码，给出无效输出
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("./codegeex4-all-9b/", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
"./codegeex4-all-9b/",
torch_dtype=torch.bfloat16,
low_cpu_mem_usage=True,
trust_remote_code=True
).to(device).eval()
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True ).to(device)
with torch.no_grad():
outputs = model.generate(**inputs, max_length=256)
outputs = outputs[:, inputs['input_ids'].shape[1]:]
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:01<00:00, 2.87it/s]
<|system|>
你是一位智能编程助手，你叫CodeGeeX。你会为用户回答关于编程、代码、计算机方面的任何问题，并提供格式规范、可以执行、准确安全的代码，并在必要时提供详细的解释。<|user|>
write a quick sort<|assistant|>
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I am getting this error after running the example code.
This is the error-
~.cache\huggingface\modules\transformers_modules\THUDM\codegeex4-all-9b\6ee90cf42fbd24807825b5ff6bed9830a5a4cfb2\modeling_chatglm.py in forward(self, hidden_states, attention_mask, rotary_pos_emb, kv_cache, use_cache)
492 # adjust key and value for inference
493 if kv_cache is not None:
--> 494 cache_k, cache_v = kv_cache
495 key_layer = torch.cat((cache_k, key_layer), dim=2)
496 value_layer = torch.cat((cache_v, value_layer), dim=2)
ValueError: too many values to unpack (expected 2)