both llama.cpp and vllm support inference with rocm or vulkan.

inference is the easiest thing to decouple from nvidia.