Remove 240s request timeout on Inference API and make docs on parameters like reasoning effort clearer
Currently the 240s server-side timeout on all requests makes it difficult to seriously use the gateway especially for models with a low throughput, and I’m not certain if the gateway forwards parameters like effort for Anthropic & OpenAI models either?
Log in to comment and vote
No comments yet
Be the first to share your thoughts.