Administration

Operations and Runtime: capacity, rate limits, and acceptance (administrators and operators only)

Evaluate operating capacity

For administrators, implementation teams, and operators.

Before launch, estimate how many AOS Work requests, ordinary Runtime tasks, and Sandbox tasks your infrastructure can handle. Check whether model rate limits constrain expected throughput, and use load tests to verify monitoring and recovery procedures.

The figures below are planning baselines for separate AOS Work and Runtime hosts with approximately 15%–25% production headroom. They are not a customer-environment commitment. Final capacity must be proven with target-environment tests and actual model quotas.

Quick pre-launch check

  1. Run representative load tests in the target environment and record success rate, response time, and resource usage.
  2. Confirm model quotas and rate limits cover the expected traffic.
  3. Check that CPU, memory, and disk still have enough headroom during peak load.
  4. Prepare an executable recovery plan for failures, rate limiting, or service interruptions.

Evaluation measures

MeasurePurposeAcceptance use
QPS / req/sAOS Work interface requests per secondStable under target load without persistent errors or saturation
Run/sOrdinary Runtime tasks per second without SandboxValidate with actual model calls and external dependencies
Session/sComplete AI tasks per second with SandboxMeasure light, medium, and heavy work separately
Success rateSuccessful requests/tasks divided by all requests/tasksCapacity conclusion requires at least 99%
P50 / P95 / P99Median, typical-tail, and long-tail latencyRecord all three; set thresholds through the business SLA
CPU, memory, diskService and Sandbox headroomRetain safe headroom during peak tests; CPU count alone is not concurrency

Review usage and execution quality

Open 【Overview】 → 【Usage Overview】, select a time range and refresh, then review activity, token and Credits consumption, data coverage, and execution quality. This example uses local sample data. Usage metrics do not replace QPS, latency, CPU, and memory load tests in the target environment.

Infrastructure capacity baseline

MachineStable AOS Work requestsOrdinary Runtime without SandboxSandbox light / medium / heavyRecommended use
4 vCPU / 16 GB180–250 req/s20–35 Run/s1–3 / 0.5–1.5 / 0.1–0.5 Session/sDevelopment, demos, low traffic
8 vCPU / 32 GB320–430 req/s45–75 Run/s3–7 / 1–3 / 0.2–0.8 Session/sSmall-to-medium production, light AI tools
16 vCPU / 64 GB450–500 req/s140–160 Run/s8–20 / 3–8 / 0.5–2 Session/sCurrent recommended production baseline
32 vCPU / 128 GB480–600 req/s220–300 Run/s16–36 / 6–15 / 1–4 Session/sLarger production, more concurrent tool tasks

Light Sandbox work performs limited state queries and file I/O; medium work includes roughly 8 MB file writes and multi-round calculation; heavy work includes roughly 32 MB writes and intensive repeated calculation. Long-running commands, large files, complex code, and document generation should be planned as medium or heavy work.

Deployment and concurrency baseline

MachineWithout SandboxWith SandboxPrimary risk
4 vCPU / 16 GB1 Runtime, 16–32 ordinary concurrent tasksLight 2; medium/heavy 1CPU and memory headroom
8 vCPU / 32 GB1–2 Runtimes, about 32 ordinary concurrent tasks eachLight 4–6; medium 2–3; heavy 1–2Disk and memory under medium/heavy work
16 vCPU / 64 GB4 Runtimes, maximum 64 ordinary concurrent tasks each4 Runtimes; light 12, medium 6, heavy 3CPU contention when co-hosted with AOS Work
32 vCPU / 128 GB4–8 Runtimes, maximum 64 ordinary concurrent tasks each8 Runtimes; light 24, medium 12, heavy 6Database, messaging and disk throughput limits

Never apply the no-Sandbox concurrency limit directly to Sandbox work. Prove absence of CPU, memory, and I/O contention if AOS Work and Runtime share a host.

Include model limits in capacity calculations

End-to-end throughput is the minimum of AOS Work capacity, Runtime/Sandbox capacity, model RPM/RPS, model TPM/TPS, and model concurrency or provisioned throughput.

Record applicable limits separately for each platform account/subscription, region, model, version, and endpoint/deployment type. Context Window and Credits do not replace provider rate limits.

Required RPM = peak business requests/s × 60 × model calls per business request
Required TPM = peak business requests/s × 60 × (P95 input tokens + P95 output tokens) × provider conversion factor

Plan sustained production load at only 70%–80% of provider quota to reserve capacity for bursts, retries, and long-tail output. Current document-generation observations are approximately 100,000–500,000 cumulative tokens per complete task; use 500,000 as an initial upper bound when samples are unavailable, then replace it with measured P50/P95 values.

Runtime System Tool and Action boundary

Discover business actions using workbench_action_search, then execute the Action Key and Schema returned for that run. Search is neither authorization nor business-completion evidence. access=user is limited to current-user-accessible data; admin actions also require an administrator or super-administrator seat.

Implementation references