Planning
AOS Work and Runtime Capacity Planning Summary
| Term | Plain-language explanation |
|---|
| AOS Work | The backend service for user login, page access, Workspace reads, Credits, and other business APIs. |
| Runtime | The backend service that actually receives and executes AI tasks. |
| Sandbox | An isolated runtime environment created for AI tool tasks. It can execute commands and read or write files, but consumes more CPU, memory, and disk. |
| No Sandbox | No isolated environment is created and no file or command tools are executed; this tests only Runtime's basic task-processing capability. |
| QPS / req/s | The number of API requests that can be processed per second. |
| Run/s | The number of ordinary tasks Runtime can process per second. |
| Session/s | After Sandbox is enabled, the number of complete AI tasks that can be completed per second. A complete task may contain multiple API, model, and tool calls. |
| P95 | 95% of requests complete within this time; used to observe the experience of most users. |
| P99 | 99% of requests complete within this time; used to observe a small number of slow requests and tail latency. |
| Success rate | The proportion of successfully completed requests or tasks among all requests. Capacity conclusions require a success rate of at least 99%. |
| Runtime service instance | One running Runtime service container. Multiple instances can share the task load. |
| Concurrent task count | The maximum number of tasks one Runtime service executes at the same time; it is not the number of CPU cores. |
The following table supports machine selection. It assumes that AOS Work and Runtime use separate dedicated machines, with approximately 15%–25% production safety capacity reserved. Sandbox requires more memory, so the memory configurations are intentionally conservative.
| Machine size | Stable AOS Work API volume | Runtime ordinary tasks (Sandbox disabled) | Light Sandbox tasks | Medium Sandbox tasks | Heavy Sandbox tasks | Applicable scenario |
|---|
| 4 cores / 16 GB | 180–250 requests/s | 20–35 tasks/s | 1–3 tasks/s | 0.5–1.5 tasks/s | 0.1–0.5 tasks/s | Development, testing, demonstrations, low-traffic business |
| 8 cores / 32 GB | 320–430 requests/s | 45–75 tasks/s | 3–7 tasks/s | 1–3 tasks/s | 0.2–0.8 tasks/s | Small- to medium-scale production, lightweight AI tools |
| 16 cores / 64 GB | 450–500 requests/s | 140–160 tasks/s | 8–20 tasks/s | 3–8 tasks/s | 0.5–2 tasks/s | Current recommended production baseline |
| 32 cores / 128 GB | 480–600 requests/s | 220–300 tasks/s | 16–36 tasks/s | 6–15 tasks/s | 1–4 tasks/s | Medium- to large-scale production, many concurrent tool tasks |
| Machine size | AOS Work recommendation | Runtime ordinary-task recommendation | Runtime Sandbox recommendation | Main resource risk |
|---|
| 4 cores / 16 GB | 1 AOS Work service | 1 Runtime service, executing 16–32 ordinary tasks simultaneously | 1 Runtime service; concurrency 2 for light tasks and 1 for medium/heavy tasks | Limited CPU and memory headroom; unsuitable for heavy tasks |
| 8 cores / 32 GB | 1 AOS Work service | 1–2 Runtime services, each executing about 32 ordinary tasks simultaneously | 1–2 Runtime services; concurrency 4–6 for light tasks, 2–3 for medium tasks, and 1–2 for heavy tasks | Monitor disk and memory during medium/heavy-task concurrency |
| 16 cores / 64 GB | 1 AOS Work service | 4 Runtime services, each executing up to 64 ordinary tasks simultaneously | 4 Runtime services; concurrency 12 for light tasks, 6 for medium tasks, and 3 for heavy tasks | AOS Work and Runtime compete for CPU when colocated |
| 32 cores / 128 GB | A multi-instance AOS Work design still requires validation | 4–8 Runtime services, each executing up to 64 ordinary tasks simultaneously | 8 Runtime services; concurrency 24 for light tasks, 12 for medium tasks, and 6 for heavy tasks | Database, messaging system, and disk may reach their limits before CPU |
| Task type | Example | Main consumption |
|---|
| Light task | Query runtime status without extensive file I/O | Model, task scheduling, and database |
| Medium task | Write an approximately 8 MB file and perform multiple rounds of computation | CPU, disk, and isolated-environment calls |
| Heavy task | Write an approximately 32 MB file and perform intensive repeated computation | CPU, disk, and resource contention between multiple isolated environments |
For real business workloads that run commands for a long time, process large files, generate documents, or execute complex code, plan capacity using the medium- or heavy-task profile.
| Measure | Purpose | Acceptance use |
|---|
| QPS / req/s | AOS Work interface requests per second | Stable under target load without persistent errors or saturation |
| Run/s | Ordinary Runtime tasks per second without Sandbox | Validate with actual model calls and external dependencies |
| Session/s | Complete AI tasks per second with Sandbox | Measure light, medium, and heavy work separately |
| Success rate | Successful requests/tasks divided by all requests/tasks | Capacity conclusion requires at least 99% |
| P50 / P95 / P99 | Median, typical-tail, and long-tail latency | Record all three; set thresholds through the business SLA |
| CPU, memory, disk | Service and Sandbox headroom | Retain safe headroom during peak tests; CPU count alone is not concurrency |
These figures are planning references, not customer-environment commitments. Validate final capacity with target-environment tests and actual model quotas. Never apply no-Sandbox concurrency limits directly to Sandbox work; verify CPU, memory, and I/O contention when services share a host.