KN001: Measuring the Background Rate Limit of Apple’s On-Device Foundation Model
How Apple’s on-device Foundation Models limit apps running in the background, and how Karui copes.
Overview
Karui is a private iPhone notification filter built on iOS 27’s new notification trigger.
iOS 27’s “When I receive a notification” Shortcuts trigger runs an automation for each notification from the apps you choose. Karui’s Shortcut automation calls an App Intent that runs in the background. The intent builds a prompt and makes one guided-generation request with LanguageModelSession:
let decision = try await session.respond(
to: prompt,
generating: TriageDecision.self
)With Karui on screen this takes about 4 seconds, but in the background the request is often refused.
Apple documents that the limit applies only in the background, but not how large it is or how it recovers. The measurements below come from one iPhone’s logs over a few days, and a month of its History.
Apple’s TN3193 covers the model’s context window. This note covers how often an app can use the model in the background in practice.
What the limit looks like
Refusals arrive as the rateLimited generation error in under 35 milliseconds, too fast for the model to have run. Apple’s documentation for the error says:
“This error will only happen if your app is running in the background and exceeds the system defined rate limit.”
Karui’s logs showed the same pattern each time, a burst of accepted calls followed by hours of refusals.
| Window | Accepted | Then refused |
|---|---|---|
| 10:37–11:16 PM | 8 calls | 11:48 PM to 5:13 AM |
| 2:44–2:50 PM | About 7 calls | 3:03 to 4:59 PM |
| 5:54–6:32 PM | 8 calls | 7:59 to 9:49 PM |
All calls were in the background, with the iPhone unplugged. Three slow calls of 8 to 15 seconds got through after midnight on the first night.
In one case, refusals continued across a 74-minute gap, so I assume this isn’t a simple hourly quota.
A month of History
Karui’s History records what made each notification’s first sort. In its export from September 20 to October 21, 2026:
- 1,651 of 3,443 on-device requests (48%) were refused as rate limited. Across the 31 days, the median day had half its requests refused.
- Before each refusal, the model had accepted a median of 6 requests in a row, over 193 bursts. The middle half accepted 3 to 11.
- Refusals went on for a median of about 40 minutes before a request was accepted again, and 80 of 192 stretches lasted an hour or more. The longest was 49 refusals in a row over 16 hours, never more than 1.7 hours apart.
- Private Cloud Compute answered 1,285 of the 1,651 refused notifications (78%) through the shortcut. Karui sorted most of the rest again once it was opened.
The export doesn’t say whether Karui was on screen for a request, so some accepted requests were probably made in the foreground, which would make bursts look longer than they were. Karui 0.2.3 records the app’s state with each sort, so later exports can tell them apart. A stretch is measured to the next accepted request, so it also depends on when notifications arrived.
Factors that did and didn’t affect refusals
Karui logs the device’s state with every model call. Refusals happened with the iPhone locked and unlocked, at nominal thermal state, with Low Power Mode off, and at 60 to 100 percent battery.
As Apple’s documentation says, calls with Karui on screen weren’t affected.
One night on the charger, all 10 background calls between 1:30 and 5:51 AM succeeded. They were also spread out, about two an hour. The first refusal came 15 minutes after the iPhone was unplugged.
In a later log, calls failed the same way in Low Power Mode at 20 percent battery, in under 20 milliseconds.
Calls that got through took a median of about 3 seconds with Karui on screen (19 calls) and about 4 seconds in the background (57 calls). The slowest took 12.5 seconds.
Misleading symptoms
The error message
The rate-limit error message suggests using non-streaming requests in background activities. Karui never streams though.
Timeouts that are really suspensions
One call “timed out” after 70 seconds against a 20-second limit. ContinuousClock keeps counting while the process is suspended, and iOS had suspended Karui mid-call.
A later log showed the same thing at a larger scale. Two calls started with Karui on screen reported timing out after 25 and 46 minutes. Each timeout was logged in the same millisecond iOS resumed Karui, so the limit only fired once Karui could run again.
Other failures
Rarely, a call fails with an error from the safety layer’s sensitive content analysis instead.
Coping
Karui still delivers every notification when the model refuses:
- It falls back to hard-coded rules. A deterministic classifier decides right away. Important contacts, mentions, one-time codes, and urgent terms were already decided by rules before the model ran.
- It rechecks once it comes to the foreground. Notifications decided by rules because of a rate limit are marked and reclassified the next time Karui is opened.
- It tries BGProcessingTask. Karui schedules one after a rate limit, and iOS ran them 15 minutes to 2 hours later, even unplugged. Whether model calls from a background task get a larger budget is still unknown.
- It asks Private Cloud Compute instead, through the user’s shortcut. The triage intent returns a flag and a ready-made prompt, the shortcut’s Use Model action sends the prompt to Private Cloud Compute, and a second intent applies the answer. In Karui’s logs, all 8 rate-limited notifications one morning got an answer 2 to 3 seconds after the on-device refusal. For how Karui builds on this, see KN002.
Private Cloud Compute has its own per-device limit, which Karui’s former Always mode reached after about 10 requests in an hour and a quarter. KN002 covers that limit and how Karui backs off.
The limit also covers an app’s own calls to Private Cloud Compute. While Karui’s on-device model was rate limited, all 64 of its direct calls to PrivateCloudComputeLanguageModel were refused within milliseconds, but the shortcut’s Use Model step, under Shortcuts’ own budget, answered more than half. The shortcut is the way around this limit; KN002 has the measurements.
Revision History
- 2026-10-22 Added a month of History: how often requests were refused, how many got through first, how long refusals lasted, and how often Private Cloud Compute answered instead.
- 2026-10-19 Added that the background limit also covers an app’s own calls to Private Cloud Compute.
- 2026-09-30 Added refusals in Low Power Mode, how long successful calls take, and much longer timeouts caused by suspension.
- 2026-09-29 Retitled, noted Private Cloud Compute’s own limit, and improved the Japanese translation.
- 2026-09-28 First published.