AI inputs, outputs and tool calls not recorded as redacted, retained security telemetry

Trace every model and tool call as a security event, and keep raw prompt and output text out of general logs.

Instrument the model client and the agent's tool executor once, where every call passes: OpenTelemetry GenAI instrumentation (opentelemetry-instrumentation-openai-v2, OpenLLMetry Traceloop.init(), OpenInference), Langfuse (@observe or langfuse.openai) or LangSmith tracing, or an audit-log call in the tool dispatcher. Record the caller (user or agent identity), time, model, tool names and arguments, guard verdicts and token counts. Turn message content capture off or pass it through a redaction step (OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false, Langfuse mask=, LangSmith hide_inputs), never write raw prompts, messages or completions to application loggers or print statements, set a retention period on the telemetry store, restrict who can read it, and write security events to append-only or signed storage so an attacker who gains access cannot erase their trail.

Where it goes: 1 application source code, 3 config and feature flags, 10 logs and telemetry, 15 agent action surface.

What reviewers look for: an instrumentor, tracing decorator or audit-log call that covers every model and tool call; content capture disabled or a redaction function applied before export; no logger.info / console.log of prompt, messages, completion or user input variables; a retention setting and an access policy for the telemetry store.

logger.info(f"prompt={messages} reply={completion.choices[0].message.content}")

# once at startup
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
OpenAIInstrumentor().instrument()   # gen_ai.* spans for every call
# environment: OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false

logger.info('chat call', extra={'user': user_id, 'model': MODEL,
            'tokens': completion.usage.total_tokens})

Engineering guidance, not legal advice. Each provision below may add its own details (a cadence, a deadline, a required notice element): open it for those.