From what I understand, it's from people using it in their workflows - say, Claude but keep hitting the rate limits, so they have to wait until Claude says "you got 10 messages left until 9pm", so when they hit that, or before they switch to (maybe) ChatGPT manually.
With the router thingy, it keeps a record, so you know every query where you stand, and can switch to another model automatically instead of interrupting workflow?
I may be explaining this very badly, but I think that's one use-case for how these LLM Routers help.
We get rate limited when using Azure's OpenAI API. As a gov contractor working with AI, I have limited means for getting access to frontier LLMs. So routing tools that can fail over to another model can be useful.
With the router thingy, it keeps a record, so you know every query where you stand, and can switch to another model automatically instead of interrupting workflow?
I may be explaining this very badly, but I think that's one use-case for how these LLM Routers help.