Rate Limiting | SheCodes
Advanced 5 min read

Rate Limiting for AI Coders

The cheap insurance that stops one bad actor from emptying your wallet.

In this chapter

  • What rate limiting is and why every app with an AI feature needs it
  • The 3 real disasters that happen without rate limits
  • How to ask AI to add rate limits: exact prompts you can copy
  • What limits to pick for different parts of your app

What Is Rate Limiting?

Imagine you run a coffee shop with one barista. If 200 people walk in at once, the barista can't handle it. Your service crashes. So you put a sign up: "Maximum 5 drinks per customer per hour."

Rate limiting is the same thing for your app. A cap on how many times one person can use a feature in a window of time. After they hit the cap, they're politely told to wait.

Why You Need This (3 Real Stories)

Here are three things that happen to AI coders all the time when they don't set limits:

The viral $400 bill You launch your AI app on Product Hunt. A bot finds your "ask the AI" button and clicks it 30,000 times in an hour. Each click costs you a few cents in OpenAI fees. You wake up to a $400 bill.
The runaway chatbot One user is having fun and sends the AI 500 messages over the weekend. That's $50 in API costs from one person. Multiply by 10 power users and you've burned half your monthly budget on a few people.
The dependency disaster Your app pulls weather data from a free external API. A bug in your code retries the call every 100ms. You hit the API's limit and they ban your account. Your whole app breaks for everyone until you beg them to let you back in.

All three are preventable. Add rate limits and none of these happen.

Where It Matters Most

You don't need limits on every button. You need them on the parts of your app that cost you money or could be abused:

AI features CRITICAL Anything that calls OpenAI, Anthropic, or any AI provider. Every call costs money. This is the #1 place limits matter.
External APIs HIGH Weather data, currency conversion, image search, anything you don't host yourself. Most have their own limits: go over and you get blocked.
Login & signup HIGH Without limits, hackers can try millions of password combinations. Stop them at 5 attempts per minute.
Email sending MEDIUM If your app sends emails (welcome emails, reminders), limit them so spam attacks don't get your email account banned.

How to Ask AI to Add Rate Limits

You don't write the code. You describe what you want and AI handles the technical setup. The trick is to be specific about three things in your prompt:

The 3 things to tell AI

  1. What feature to limit: "the AI chat", "the signup form", "the search bar"
  2. How many uses per time window: "20 times per hour", "5 attempts per minute"
  3. What to show when the limit is hit: "a friendly message saying try again later"
THE STARTER PROMPT

β€œAdd rate limiting to my app. I want to limit my AI chat feature so each user can only send 20 messages per hour. If they hit the limit, show a friendly message: "You've hit your hourly limit, try again in [X minutes]." Use the simplest setup that works on Vercel: pick whatever free tool you recommend. Walk me through where to get any keys I need.”

Picking the Right Limits

The right number depends on the feature. Too low = annoying. Too high = pointless. Here are sensible starting points you can give AI:

AI chat (free users) 10-20 per hour Enough for normal use, blocks runaway loops and bots
AI chat (paying users) 100-500 per hour Generous but still bounded
Login attempts 5 per minute Real users only try once or twice. Anything more is brute force
Password reset emails 3 per hour Stops spamming people's inboxes
Public contact form 3 per hour per IP Stops spam without blocking real customers
External API calls Whatever the API allows, divided by 2 Stay under their limit so you never get blocked

Show a Friendly Message, Not an Error

When someone hits the limit, the worst thing you can do is show a scary error like "Error 429: Too Many Requests." Real users have no idea what that means. They just think your app is broken.

❌ Confusing

  • Error 429: Too Many Requests

βœ… Friendly

Whoa, slow down! You've hit your hourly limit. Try again in 23 minutes.

MAKE LIMIT MESSAGES FRIENDLY

β€œWhen my app hits a rate limit, show a friendly toast notification instead of a scary error. The message should explain what happened in plain English ("You've used your hourly limit") and tell the user when they can try again. If the user is on a free plan, also offer them an "Upgrade for higher limits" link.”

Common Gotchas

I locked myself out while testing

Tell AI: "skip the rate limit when running on localhost so I don't lock myself out during development."

Limits are too tight, real users complain

Doubling the limit is a one-line change. Tell AI "raise the AI chat limit from 20 to 50 per hour" and you're done.

Limits don't work in production

You probably forgot to add the API keys for the rate-limit service to Vercel. Settings β†’ Environment Variables.

Same user gets a different bucket on each device

AI is using IP address as the key. Tell it to use the user's account ID instead so the limit follows them across devices.

Real World Example: Protecting your AI chat the day before launch

You're launching your AI image generator on Product Hunt tomorrow. You're paying for OpenAI on a credit card with a $50 limit. Without rate limits, one viral hour could blow past it and your card gets declined mid-launch. With limits set to 5 generations per hour for free users, even 10,000 visitors can't break the bank: the math just works out.

Build this with AI

β€œI'm launching my AI image generator tomorrow. Add rate limits to protect my OpenAI bill. Free users (anyone not logged in): 3 image generations per hour, tracked by their IP. Logged-in users: 10 generations per hour, tracked by their account. Show a friendly toast when they hit the limit explaining when they can try again. Use the simplest free setup: pick whatever you recommend for Vercel. Make sure it's skipped on localhost so I don't lock myself out testing it.”

Prompt This

LIMIT MY AI FEATURE

β€œMy app has an AI chat that calls OpenAI. Add a rate limit so each user can only send 20 messages per hour. If they hit the limit, show a friendly message and offer them an upgrade link. Skip the limit on localhost. Use whatever free tool you recommend.”

LIMIT MY EXTERNAL API

β€œMy app calls the [API name] API to fetch [data type]. Their free tier allows 1000 requests per day. Add caching so I don't call the API more than necessary, and a rate limit so my app never goes over their daily cap even if traffic spikes.”

LIMIT LOGIN ATTEMPTS

β€œOn my login page, add a rate limit: 5 attempts per minute per IP. After 5 failed attempts, show a friendly message saying "Too many tries. Wait a minute and try again." This protects against people guessing passwords.”

Frequently Asked Questions

What is rate limiting in plain English?

It's a cap on how many times someone can use a feature in a window of time. 'Free users can ask the AI 10 questions per hour.' After they hit the limit, they're politely told to wait. It's like a bouncer at a club making sure no one orders 200 drinks at once.

Why can't I just trust users?

Most users are great. But the internet has bots. And sometimes a real user clicks a button 50 times in a panic. And sometimes an honest mistake in your code creates an infinite loop. Rate limits protect you from all three: you don't need to assume bad intent, just to have a safety net.

Will rate limits annoy my real users?

Not if you set them right. A normal user might use your AI chat 5-10 times in an hour. A limit of 30/hour is invisible to them but stops abuse. The trick is to set the limit high enough that 99% of real users never notice it.

Do I need rate limits on day one?

If your app calls a paid API (OpenAI, Anthropic, etc.), yes: set them up before launch. If you're just storing data, you can wait until you have your first 100 users. The rule of thumb: anything that costs you money per use needs a limit before launch.