Run falcon on Mac
I tried to run the model on Mac Studio. It's said that MPS doesn't support bfloat16, so I changed the dtype in the sample code to float16. And I got the error:
loc("varianceEps"("(mpsFileLoc): /AppleInternal/Library/BuildRoots/ff32e6fb-db00-11ed-a068-428477786501/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShadersGraph/mpsgraph/MetalPerformanceShadersGraph/Core/Files/MPSGraphUtilities.mm":228:0)): error: input types 'tensor<1x61x1xf16>' and 'tensor<1xf32>' are not broadcast compatible
LLVM ERROR: Failed to infer result type(s).
What should I do to make it work?
I'm getting the same error on a different model
I success to run the model when I change bfloat16 to float32, it seems that the MPS doesn't support some dtype and some dtype doesn't compatible to each other.