Despite some limitations, you will probably find Bumblebee (https://github.com/elixir-nx/bumblebee) interesting.

"Bumblebee provides pre-trained Neural Network models on top of Axon. It includes integration with HuggingFace Models, allowing anyone to download and perform Machine Learning tasks with few lines of code"

Ortex exists to run ONNX models: https://github.com/elixir-nx/ortex

Not directly but it supports the ONNX Runtime and has support for many of the models you might want through Bumblebee and Hugging Face

https://github.com/elixir-nx/bumblebee

(sorry for the double post!)