THUDM/chatglm2-6b accuracy=0 on lambada_openai dataset with lm-eval.

#97

by changwangss - opened Nov 2, 2023

Nov 2, 2023

THUDM/chatglm2-6b accuracy=0 on lambada_openai dataset with lm-eval.
Here is command, Could you have a try?

git clone https://github.com/EleutherAI/lm-evaluation-harness.git
cd lm-evlauation-harness
python setup.py install

python main.py --model hf-causal --model_args pretrained=THUDM/chatglm2-6b,trust_remote_code=True  --tasks lambada_openai --limit 10 --batch_size 1 --no_cache

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment