Any inference app with UI for onnx models?
Hello, is there any inference app with UI for onnx models? Or at least any inference app that can act as OpenAI compatible API server?
ONNX is currently mostly used for edge or web inference. For a OpenAI-compatible server or UI check the gguf or mlx versions (you can use them with LMStudio and many others)
I released a WebGPU demo that runs 100% locally in your browser, if that's what you're looking for? :) https://huggingface.co/spaces/webml-community/Zyphra-ZR1-WebGPU
I released a WebGPU demo that runs 100% locally in your browser, if that's what you're looking for? :) https://huggingface.co/spaces/webml-community/Zyphra-ZR1-WebGPU
Thanks, that's nice. Seeing a complete inference with GUI is nice, but I don't suppose it allows to switch to different models easily? I'm looking for some universal inference solution for onnx models. That means including, but not limited to this one. Basically an alternative to aforementioned GGUF models running in LM Studio. Preferably something that could be used as OpenAI compatible API server only for onnx models, so that I could connect to it from UIs of my choice.
Ah I see. For an OpenAI compatible API, maybe make a feature request to https://github.com/microsoft/onnxruntime-genai? There's been a similar thread here: https://github.com/microsoft/onnxruntime-genai/discussions/419, and looks like a community member has started creating one: https://github.com/bpietroiu/GenAI.Server.