Is it possible to compile this model while using flash_attn_2?
#81
by Thomas2419 - opened
As title says, due to the dynamic nature of the padding and unpadding it kept failing for me, but im wondering if I was doing something wrong or if there's a way to effectively compile this model?