Smart Queue: Automatic Rate Limit Management

AI providers impose rate limits — caps on how many requests you can make per minute, per hour, or per day. When you're running multiple agents or batching large tasks, you'll hit them. The Smart Queue handles this automatically so you don't have to.

The Rate Limit Problem

Without queue management, hitting a rate limit means one of two things: your task fails and you have to restart it, or you implement a bunch of retry logic yourself. Neither is great when you're trying to get work done.

The Smart Queue sits between your task requests and the AI provider. When a task can run immediately, it does. When rate limits are active, the queue holds the task and releases it as soon as capacity is available — automatically, with no failed requests and no manual retries.

How the Queue Works

When you submit tasks to SuperBuilder, each one enters the queue with its priority and requirements:

1. Available capacity — if the provider has capacity, the task runs immediately
2. Rate limited — if you're at your limit, the task waits and is released when the window resets
3. Retry on failure — if a task fails due to a transient error, the queue retries it with exponential backoff
4. Priority ordering — high-priority tasks (like interactive threads) are processed before background tasks

The queue is persistent. If you close SuperBuilder and reopen it, queued tasks resume. If the rate limit window resets overnight, queued tasks automatically run in the morning.

Batch Task Queuing

The Smart Queue is most powerful when you're running many tasks in one go. Common patterns:

Repository-wide tasks — "add JSDoc comments to all 200 functions in this repo" queues as 200 individual tasks. They run as fast as your rate limits allow, completing the whole job without manual intervention.

Test generation — "write tests for every file in the /services directory" fans out into one task per file. The queue drains them in order, respecting limits.

Documentation runs — "generate a README for each package in this monorepo" — same pattern. Queue everything, walk away.

Priority Levels

Tasks in the queue have priority levels:

• Interactive — tasks you're actively working on in a thread. These always go to the front of the queue.
• Standard — regular agent tasks. Run in order.
• Background — scheduled or batch tasks. Run when there's spare capacity.

You can configure the default priority for different task types, or set it per-task when you queue work.

Queue Visibility

The SuperBuilder interface shows you:

• How many tasks are currently queued
• Estimated wait time based on rate limit windows
• Which tasks are running vs waiting
• Queue position for each pending task

You're never guessing whether a task is running or stuck. The queue state is always visible.

Rate Limit Windows

Different AI providers have different rate limit structures. SuperBuilder tracks them per provider:

• Per-minute limits — short windows that reset quickly (usually under a minute)
• Per-hour limits — longer windows that may require pausing for up to an hour
• Per-day limits — daily caps that reset at midnight UTC

The Smart Queue knows each provider's limit structure and schedules tasks accordingly. It's smart enough to fill capacity to the edge of the limit without going over.

Combining Queue with Scheduler

The Smart Queue and Agent Scheduler work together naturally. You can schedule a large batch task to start overnight, and the queue handles rate limits throughout the run without you having to intervene.

Example: schedule "run tests for the entire test suite and fix every failure" at 11pm. The scheduler kicks it off, the queue manages rate limits through the night, and you wake up to finished work.

Frequently Asked Questions

Does the queue work across all AI providers?

Yes. SuperBuilder tracks rate limits per provider and manages queuing for each one independently. If you're using multiple providers, the queue handles each correctly.

What happens if a task in the queue fails with a non-rate-limit error?

Non-rate-limit errors (like a malformed request) don't retry automatically — they surface to you for inspection. Rate limit errors and transient network errors retry automatically.

Can I cancel tasks that are in the queue?

Yes. You can cancel any queued task from the queue view before it starts running.

Can I pause the queue?

Yes. You can pause the queue at any time, which stops new tasks from starting while letting the current task finish. This is useful if you want to review what's queued before it runs.

Does the queue work when SuperBuilder is closed?

For cloud tasks, yes — the queue runs on SuperBuilder's servers. For local tasks, the queue persists and resumes when you reopen SuperBuilder.