AI usage, agent steps, and spend not bounded

Cap agent steps and output tokens on every model call, set per-key and per-project budgets with usage alerts, and issue scoped, expiring model keys.

Bound work at three layers. In code, every agent or tool-use loop has a hard step cap (a counted for loop, max_iterations, recursion_limit, maxSteps or stopWhen) and every user-triggered call sets an output-token cap. At the gateway or provider, each key and team has a budget and rate limits (max_budget, budget_duration, tpm_limit, rpm_limit), and keys are scoped to the models and project that need them and expire on a rotation schedule. In the infrastructure that provisions the model account, a cloud budget with alert notifications surfaces anomalous spend to the owning team.

Where it goes: 1 application source code, 3 config and feature flags, 4 infrastructure-as-code, 8 model configuration.

What reviewers look for: no while True or for(;;) loop around a model call without a step counter or budget check, and no max_iterations, maxSteps, or recursion_limit set to None, Infinity, or a huge number; max_tokens, max_completion_tokens, or maxOutputTokens on user-triggered calls; gateway keys and teams with max_budget, budget_duration, and tpm/rpm limits; an aws_budgets_budget, azurerm_consumption_budget_*, or google_billing_budget with notifications in the IaC for the AI account.

while True:
    resp = client.chat.completions.create(model=MODEL, messages=msgs, tools=TOOLS)
    msg = resp.choices[0].message
    if not msg.tool_calls:
        break
    msgs += [msg, *run_tools(msg.tool_calls)]

MAX_STEPS = 10
for step in range(MAX_STEPS):
    resp = client.chat.completions.create(model=MODEL, messages=msgs, tools=TOOLS,
                                          max_completion_tokens=1000)
    msg = resp.choices[0].message
    if not msg.tool_calls:
        break
    msgs += [msg, *run_tools(msg.tool_calls)]
else:
    raise StepLimitExceeded(f'agent stopped after {MAX_STEPS} steps')

Engineering guidance, not legal advice. Each provision below may add its own details (a cadence, a deadline, a required notice element): open it for those.