Wow, that was fast! Ill try to run it tomorrow, there is a branch in the repo?
Discussion
It's in main. I'm using the llmlite lib as it also supports other backends which comes in handy. You find it in tasks/textgeneration_llmlite.py. You can also run the task files alone without the bot/other DVMs by just running it from the main folder. python tasks/nameofscript.py. let me know if I can assist with anything. Still in the process of making everything smooth for people to use.