Automatos has no per-mission budget enforcement. Cost data flows from LLM responses into the llm_usage table (via UsageTracker), and analytics endpoints surface spending trends, but nothing blocks a mission from spending beyond any limit. The platform records what was spent — it never prevents overspending.
Gap
Impact
No pre-call budget check
A runaway mission can exhaust workspace LLM credits in minutes
No per-mission cost cap
Coordinator-spawned tasks have no aggregate spending boundary
No tool policy layering
Every agent gets every assigned tool — no mission-scoped restrictions
No approval gates beyond chat
Complex missions auto-execute with no human checkpoint
Workspace.plan_limits JSONB exists but is never read
The schema hook for enforcement is present but unwired
Two TokenBudgetManager classes serve different purposes
modules/context/budget.py (context-window packing) vs stages/token_budget_manager.py (workflow tokens, latent bugs)
1.2 What This PRD Delivers
Budget admission gate — synchronous pre-call check that blocks LLM calls when budget is exhausted
Budget data model — budget_config and budget_spent JSONB schemas on orchestration_runs
Graduated thresholds — warn at 50%, throttle at 80%, stop at 100%
Pre-estimation algorithm — predict mission cost before execution for human approval
Post-call reconciliation — actual cost recorded with mission/task attribution
Workspace.plan_limits activation — wire the existing but unused JSONB field
2. Prior Art: Budget & Governance Patterns
2.1 System-by-System Analysis
OpenClaw 8-Stage Tool Policy Chain
OpenClaw implements an 8-stage monotonically narrowing tool policy chain. Each stage can only narrow the tool set — never expand. Deny always wins over allow. Enforcement happens at tool-set construction (tools passed to LLM tools= param), not post-hoc interception.
What we adopt: Monotonic narrowing invariant (workspace → mission → task → agent). Tool group shorthand for policy configuration. Enforcement at tool-set construction (already how get_tools_for_agent() works).
What we reject: 8 stages (overkill). No temporal/budget dimension. No per-mission scoping.
K8s ResourceQuota & Admission Control
K8s enforces resource limits through synchronous admission control — the API server returns HTTP 403 before the resource is created. Two layers: ResourceQuota (namespace aggregate) + LimitRange (per-pod defaults/maximums). Quota does not retroactively evict running workloads.
Priority sub-budgets (coordinator/verifier vs worker agents)
What we adopt: Synchronous admission gate. Hard rejection (not queuing). Two-layer limits (mission aggregate + per-task). In-flight work completes; next admission rejected.
AWS Budgets
AWS implements budget enforcement through soft caps with graduated automated actions. Up to 5 thresholds per budget. AUTOMATIC or MANUAL approval model per action. Critical lesson: AWS has no true hard cap — billing data updates every 8-12 hours. For LLM missions that exhaust budgets in seconds, this lag is fatal.
What we adopt: Graduated thresholds (warn → throttle → stop). Separate action thresholds from notification thresholds. Dual COST + USAGE tracking.
What we reject: Post-hoc billing approach. We need synchronous pre-call checks.
Token Bucket (Anthropic/Stripe Pattern)
For mission budgeting, use a cost-denominated token bucket with no refill:
Bucket capacity = mission budget in dollars
Each LLM call consumes estimated_cost from the bucket
After call, reconcile estimated vs actual cost
Refill disabled (missions have fixed, non-replenishing budgets)
LiteLLM BudgetManager
Two-phase pattern: projected_cost(model, messages, user) pre-call, update_cost(completion_response, user) post-call. This is the closest existing implementation to what we need.
What we adopt: The two-phase pattern (estimate → check → execute → reconcile).
POST /missions/{id}/budget with complete_current=true
Finish in-flight tasks, cancel remaining
7. Tool Policy Layering
7.1 Four-Tier Narrowing Model
Invariant: each tier can only narrow, never expand. The effective tool set is the intersection of all tiers.
7.2 Enforcement Point
7.3 ToolPolicy Schema
8. Workspace Plan Limits Activation
8.1 Current State
Workspace.plan_limits JSONB exists on the workspaces table but is never read by any code path.
8.2 Wiring Design
9. RBAC Extensions
9.1 New Permissions
Permission
Who Can
Description
mission:create
EDITOR, ADMIN, OWNER
Create a new mission
mission:approve
ADMIN, OWNER
Approve/reject mission plans
mission:review
EDITOR, ADMIN, OWNER
Review mission results
budget:view
VIEWER, EDITOR, ADMIN, OWNER
See budget status
budget:set
ADMIN, OWNER
Set mission budget
budget:override
OWNER
Increase budget beyond workspace limits
tool_policy:set
ADMIN, OWNER
Configure workspace tool policy
9.2 Integration with Existing RBAC
The existing permissions.py uses role-based access (OWNER/ADMIN/EDITOR/VIEWER). New permissions map to existing roles — no new permission system needed, just new permission checks in mission API endpoints.
10. Cost Attribution
10.1 Linking llm_usage to Missions
Decision: Add nullable mission_task_id FK to llm_usage.
Why nullable: Thousands of existing llm_usage rows from non-mission calls (chatbot, heartbeat, routing). These have no mission context and never will. Non-nullable would require backfill.
10.2 Request Type Extension
This enables computing:
coordination_cost_usd = SUM(cost WHERE request_type = 'coordinator')
verification_cost_usd = SUM(cost WHERE request_type = 'verifier')
task_cost_usd = SUM(cost WHERE request_type = 'mission_task')
def estimate_call_cost(
model: str,
input_tokens: int,
max_output_tokens: int,
) -> float:
"""
Estimate cost of a single LLM call.
Input tokens: exact (counted by tokenizer pre-call).
Output tokens: max_tokens × 0.7 (empirical median for agent tasks).
Why 0.7 and not 1.0 (worst case)?
Worst-case over-reserves budget. Missions would stall at 70% actual
spend because the gate thinks 100% is committed. Reconcile actual vs
estimated after each call; the running total uses real numbers.
"""
pricing = get_model_pricing(model)
estimated_output = int(max_output_tokens * 0.7)
return (
(input_tokens * pricing.input_per_1k / 1000)
+ (estimated_output * pricing.output_per_1k / 1000)
)
async def _handle_budget_exceeded(
self, run: OrchestrationRun, task: OrchestrationTask
) -> None:
"""
Called when budget gate rejects an LLM call.
K8s pattern: in-flight work completes; next admission rejected.
The current task's in-progress LLM call (if any) finishes.
No new LLM calls are allowed until budget is resolved.
"""
config = BudgetConfig(**run.budget_config)
if config.model_downgrade_enabled:
# Strategy 1: Downgrade remaining tasks to BUDGET_MODELS
remaining_tasks = await self._get_pending_tasks(run.id)
for t in remaining_tasks:
t.contractor_config["model"] = self._get_budget_model(t.task_type)
await self._emit_event(run.id, "budget_model_downgrade", {
"remaining_tasks": len(remaining_tasks),
})
# Re-estimate with cheaper models — may fit within budget
new_estimate = await self._re_estimate_remaining(run, remaining_tasks)
if new_estimate <= run.budget_remaining:
return # Downgrade resolved it
if config.approval_model == "manual":
# Strategy 2: Pause for human decision
await self._transition_run(run, RunState.BUDGET_EXCEEDED)
await self._notify_user(run, "budget_exceeded", {
"spent": run.budget_spent["cost_usd"],
"cap": config.max_cost_usd,
"options": ["increase_budget", "cancel_mission", "downgrade_models"],
})
else:
# Strategy 3: Auto-abort
await self._transition_run(run, RunState.FAILED,
reason="Budget exceeded and no resolution available")
Tier 1: Workspace Policy
Tools: [all platform_*, workspace_*, composio_*]
Deny: [workspace_exec] ← admin removed shell access
└─ Tier 2: Mission Policy
Tools: [platform_search_web, workspace_read_file, workspace_write_file]
← coordinator scoped to research/writing tools
└─ Tier 3: Task Policy
Tools: [platform_search_web, workspace_read_file]
← this specific task only needs search + read
└─ Tier 4: Agent Policy
Tools: [platform_search_web, workspace_read_file]
← agent's DB tool assignments intersected with above
← for contractors: explicit tools from coordinator
# In orchestrator/modules/tools/tool_router.py
async def get_tools_for_agent(
self,
agent_id: int,
workspace_id: int,
mission_policy: Optional[ToolPolicy] = None, # NEW
task_policy: Optional[ToolPolicy] = None, # NEW
explicit_tools: Optional[list[str]] = None, # NEW (PRD-104 contractors)
) -> list[dict]:
"""
Resolve tools for an agent with policy layering.
For roster agents: DB tools → intersect with policies
For contractors: explicit_tools → intersect with policies
"""
# Start with full available tools
if explicit_tools is not None:
# Contractor: coordinator-specified tools
base_tools = await self._resolve_tool_names(explicit_tools, workspace_id)
else:
# Roster agent: DB-backed tool assignments
base_tools = await self._get_agent_db_tools(agent_id, workspace_id)
# Apply policy layers (narrowing only)
workspace_policy = await self._get_workspace_tool_policy(workspace_id)
effective_tools = self._intersect_policies(
base_tools,
workspace_policy,
mission_policy,
task_policy,
)
return effective_tools
def _intersect_policies(
self,
tools: list[dict],
*policies: Optional[ToolPolicy],
) -> list[dict]:
"""Intersect tool set with each policy layer. None = no restriction."""
result = tools
for policy in policies:
if policy is None:
continue
if policy.allowed:
# Allowlist: keep only tools in the allowed set
allowed_names = set(policy.allowed)
result = [t for t in result if t["name"] in allowed_names]
if policy.denied:
# Denylist: remove denied tools (deny wins over allow)
denied_names = set(policy.denied)
result = [t for t in result if t["name"] not in denied_names]
return result
@dataclass(frozen=True)
class ToolPolicy:
"""Tool access policy for a specific scope."""
allowed: Optional[list[str]] = None # None = no allowlist restriction
denied: Optional[list[str]] = None # None = no denylist
groups: Optional[list[str]] = None # Tool group shorthand: "web", "filesystem", "code"
# Tool groups (shorthand for common sets)
TOOL_GROUPS: dict[str, list[str]] = {
"web": ["platform_search_web", "platform_browse_url"],
"filesystem": ["workspace_read_file", "workspace_write_file", "workspace_list_dir"],
"code": ["workspace_exec", "workspace_git"],
"search": ["platform_search_documents", "platform_search_web"],
"communication": ["platform_send_message", "platform_create_report"],
}
# In orchestrator/core/models/workspaces.py
# plan_limits JSONB already exists at line ~32-33
# Activation: workspace-level budget enforcement
PLAN_LIMITS_SCHEMA = {
"type": "object",
"properties": {
"max_monthly_cost_usd": {"type": "number"}, # Monthly spending cap
"max_concurrent_missions": {"type": "integer"}, # Max active missions
"max_mission_cost_usd": {"type": "number"}, # Per-mission default cap
"default_tool_policy": {"type": "object"}, # Tier 1 tool policy
"allowed_models": {"type": "array", "items": {"type": "string"}},
},
}
# Enforcement: checked at mission creation
async def check_workspace_limits(
workspace_id: int, proposed_budget: float
) -> None:
"""
Check workspace-level limits before creating a mission.
Raises WorkspaceLimitError if any limit would be exceeded.
"""
workspace = await get_workspace(workspace_id)
limits = workspace.plan_limits or {}
# Check monthly spending
if "max_monthly_cost_usd" in limits:
month_spend = await get_workspace_monthly_spend(workspace_id)
if month_spend + proposed_budget > limits["max_monthly_cost_usd"]:
raise WorkspaceLimitError(
f"Monthly budget would be exceeded: "
f"${month_spend:.2f} + ${proposed_budget:.2f} > "
f"${limits['max_monthly_cost_usd']:.2f}"
)
# Check concurrent missions
if "max_concurrent_missions" in limits:
active = await count_active_missions(workspace_id)
if active >= limits["max_concurrent_missions"]:
raise WorkspaceLimitError(
f"Max concurrent missions reached: {active}/{limits['max_concurrent_missions']}"
)
ALTER TABLE llm_usage ADD COLUMN mission_task_id INTEGER
REFERENCES orchestration_tasks(id) ON DELETE SET NULL;
CREATE INDEX CONCURRENTLY idx_llm_usage_mission_task
ON llm_usage(mission_task_id) WHERE mission_task_id IS NOT NULL;
