Upgrade to a newer transformers version
#39
by
the-mysh
- opened
Hi,
It would be great if it was possible to update the model to a newer version of transformers. With transformers 4.57.1, I'm getting an error that is, I believe, gone in an older version (~4.45):
File ".../modules/transformers_modules/microsoft/Phi_hyphen_3_hyphen_small_hyphen_128k_hyphen_instruct/ad85cab62be398dc90203c4377a4ccbf090fbb36/modeling_phi3_small.py", line 810, in forward
past_key_values_length = past_key_values.get_usable_length(seq_length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'DynamicCache' object has no attribute 'get_usable_length'. Did you mean: 'get_seq_length'?
Also, I can't seem to make the model work with flash attention, and I'm unable disable it in the model. I tried passing appropriate arguments to AutoModelForCausalLM.from_pretrained or even just patching it by model.config._attn_implementation = "eager", but it insists on using flash attention anyway.