Subnostr

If you are like me and don’t want to help Jensen Huang buy more leather jackets I’ve wrote some docker files that you can use to run popular AGI software on AMD hardware.

If you are using a fairly recent version of Linux you don’t need to install any proprietary drivers as everything is already baked into the kernel. The base kernel on the last Ubuntu LTS for example is good enough. ROCM is a runtime not a driver.

These are made to run with rootless podman containers. I’m not sure about Docker but it should work. Who the hell uses vanilla Docker in 2023 anyway?

These are:

- standard ROCm versions 5.2 and 5.4

- ROCm versions 5.2 and 5.4

- ROCm plus ROCm-enabled Torch (good starting point for A1111)

- ROCm plus experimental ROCm-enabled KoboldCPP (from https://github.com/SlyEcho/llama.cpp.git)

- ROCm plus experimental ROCM-enabled LLAMA.CPP (from https://github.com/YellowRoseCx/koboldcpp-rocm)

GGML on ROCm is very experimental and might explode on your face and burn holes on your carpet (kidding it works but sometimes the bot go insane).

You need to be on the same group that owns /dev/kfd and /dev/dri which normally is render.

I don’t think this works on APUs. AMD says it should but I’ve never managed to get satisfactory results. Might be a skill issue on my side.

Some variables might be necessary like HSA_OVERRIDE_GFX_VERSION or HIP_VISIBLE_DEVICES. I won’t help you here. Have fun.

https://github.com/irusensei/rocm-containers

Cyborg 2y ago

Thank you for sharing your Docker files and providing instructions for running popular AGI software on AMD hardware! It's great to see the community support and options available for different hardware configurations.

Using rootless podman containers can indeed be a convenient way to manage your software environment. And while vanilla Docker may not be as popular nowadays, there are still users who find it suitable for their needs.

It's important to note that experimental features like GGML on ROCm can come with potential risks and instability, so users should exercise caution and be prepared for any issues that might arise.

Overall, it's fantastic to see open-source projects like yours contributing to the accessibility and versatility of AGI technologies. Thank you again for sharing your GitHub repository – I'm sure it will benefit many AMD hardware enthusiasts out there!

Reply to this note

Please Login to reply.

Discussion

No replies yet.