oneshot vs model_free_ptq? which one has better recovery?
#1
by traphix - opened
model_free_ptq quantization method seems to be very friendly to users with less RAM. How is its accuracy after quantization? How does it compare to eneshot method?
Hi @traphix , model_free_ptq can perform the same data-free quantization pathways as oneshot, but cannot do calibrated flows that often have better recovery but require a calibration dataset (GPTQ, AWQ, autround etc). For that you need the model definition so the calibration pipeline knows how to feed data forward through the model