Global Feed Post Login
Replying to Avatar Nate

I personally would prefer to have the LLM separate from Amethyst, but I've personally been real interested in trying to get some of the smaller models running on my phone inside or outside Amethyst. My phone is powerful enough to run some of the models no issue, though last I checked the existing software for mobile is a bit trickier than on PC, so I'd certainly welcome an easy way to get a GPU accelerated LLM running on my phone.

Avatar
⚑ Dee Kay βš‘πŸ‡ΈπŸ‡ͺπŸ‡¬πŸ‡§πŸ‡¨πŸ‡ΏπŸ‡§πŸ‡·πŸ‡¦πŸ‡Ή 10mo ago

It's actually really easy on a modern flagship android phone.

https://play.google.com/store/apps/details?id=com.pocketpalai

The trick is to select a model optimised for your phone.

IQ4_NL versions work for me

Reply to this note

Please Login to reply.

Discussion

No replies yet.