As someone who cares about latency, why would I add an additional HTTP request to the request my users are making? I can just use an in-memory store to achieve the same thing under 1ms.
10ms is an eternity, and 100ms added on to every request is insane.
I think this is for a different kind of rate limiting. E.g. you have a website with a couple of servers, and you want to limit the number of chatGPT requests those servers are making per day. 10ms is meaningless to the chatGPT 12s p50.
You could spend an hour making your own persistence/coordination solution - or spend a minute and 5 bucks to call this service.
10ms is an eternity, and 100ms added on to every request is insane.