Oops.

Air should be also ok for smaller models, they have M2 chip. The main limitation is memory. 40G model in full fp16 precision needs 80GB, less than 40GB in q5.

This is 13B model in q5, so if you have 32GB macbook air, you should be fine. Even iPhones and iPads can technically run these models.

https://vimeo.com/837394279/ee5f723467

Reply to this note

Please Login to reply.

Discussion

Thank you for this!