Need assistance in running on Mac
Hey y'all,
I'd love to try out this model but keep running into issues with the "model family: 'Kimi_Linear' not supported". I've tried with vLLM and LM Studio but nothing works.
It seems like it requires the absolute newest version of vLLM to run, but that's only available on Linux apparently.
Can anyone provide me with a step-by-step guide to make it work on Mac? Is it even possible right now?
Would using a Docker image be one possible solution?
Any help is appreciated - I always love the Kimi models.
@x-polyglot-x Hi, looks like you may need to build a brand-new envs for vllm, especailly paying attention to the version conflicts between your conda/uv/pip pkgs.
MLX-LM has support merged: https://github.com/ml-explore/mlx-lm . Clone the main branch and install from source.
Thank you to @yzhangcs and @kernelpool -- I was able to get it working via cloning the mlx-lm branch and installing again. Thank you!