Rate limits and speed: what a price page leaves out

A rate limit caps how much you may send in a period, counted in requests, in tokens, or both. New accounts often start with low limits that rise with spending or on request. A low price is no use if the limit is below your traffic.

Read the limit for a new account, and how it is raised. If the answer is a sales call, count the days.

The first is the wait before the first token appears. It matters for chat, where a person is watching. The second is how fast tokens arrive after that. It matters for long answers and for batch work.

Both depend on the host’s hardware and how busy it is, and both change through the day. This site publishes neither, because a speed measured once is not a speed you can rely on.

Send your own typical request with streaming on. Time the first token, then count tokens per second to the end. Repeat at different hours for a few days. Do the same on each API you are comparing.

Then send requests at the rate you expect in production, and watch for errors and slowdowns. That is the test that matters.

Ask what happens when a request fails. Are failed requests billed? Is there a status page with a history? Is there a second host of the same model you can fall back to with the same client?

An OpenAI-compatible format makes the fallback simple: the request is the same, and only the base URL and key change.

Runcrate serves 124 models through one OpenAI-compatible API on one prepaid balance. You can run your speed and load tests against several models from a single account, and read the cost of the tests in one place.

This is one of a coordinated collection of Runcrate references. Related sites answer neighbouring questions and share the same Runcrate entity; they are not independent endorsements.