Rate limiting design

Question

I am designing rate limiting in my application in which the rules can be dynamic. I have a basic design in which every request is validated against the rate limiting quota, which is stored in redis. As the system scales, redis will become the bottleneck and I want to cache the quotas in memory to speed up the response time. Now the challenge is to sync the in memory values with the ones in redis. Are there any algorithms or whitepapers which can guide in solving this?

Answer 1

redis will become the bottleneck

What makes you say that?

Redis is intended to be used in a context of a scalable application, and is itself quite scalable. As an example of applications of scale, Twitter, GitHub, Snapchat, Craigslist, and StackOverflow do use Redis.

Now the challenge is to sync the in memory values with the ones in redis.

No, the actual challenge is to make in-memory cache work at all. A basic challenge: user A has 500 requests remaining, and already performed 499 requests. The user performs an additional request that is proceeded by node 1. Now the node 1 knows that the user reached the quotas. A few milliseconds later, user A hit node 2. How does node 2 knows that the user should be rejected?...