Skip to main content

Remove 240s request timeout on Inference API and make docs on parameters like reasoning effort clearer

Currently the 240s server-side timeout on all requests makes it difficult to seriously use the gateway especially for models with a low throughput, and I’m not certain if the gateway forwards parameters like effort for Anthropic & OpenAI models either?

Log in to comment and vote

No comments yet

Be the first to share your thoughts.