Global Feed Post Login
Replying to Avatar ynniv

Having followed Apple for close to three decades, I knew this would happen as soon as the first llama dropped. They tend to have a longer horizon and miss out on a lot of practical opportunities while lining up for something most people aren't thinking about yet. Given their surprising advance in unified memory and decision to make over-powered but under-featured mobile devices, preparing to run something heavy in the near future is a reasonable conclusion.

But, their advantage is going to be limited by their ecosystem. iOS is a lot more constrained than an open source OS, and while it can be a hot mess, it's also possible to lock those down too. It just takes time and code. Perhaps code written by LLMs.

Avatar
ynniv 1y ago 💬 2

Yesterday I downloaded a local LLM app to my iPhone 15 Pro Phi3 mini was throwing 15 tokens / sec. Don't get sidetracked by Apple, everyone is going the same place, and very soon.

Reply to this note

Please Login to reply.

Discussion

Avatar
Rinzler 1y ago 💬 1

Which App?

Avatar
ynniv 1y ago

This was https://github.com/guinmoon/LLMFarm

Thread collapsed
Thread collapsed
Avatar
Justin Nakamoto-San ☑️ 1y ago 💬 1

Thanks! You inspired me to search github for an android equivalent app. Found this one but I have not tested it, still downloading Q4_K_M

https://github.com/nerve-sparks/iris_android

Avatar
Justin Nakamoto-San ☑️ 1y ago

search terms I used

https://github.com/search?q=ggml+android+&type=repositories

Thread collapsed
Thread collapsed