AI Security

Continuous Monitoring for Managed AI Agents with File‑System and Shell Access

Published 2026-08-09 by AISecAll Editorial

TL;DR: Enable structured logging from the agent, ship logs to a central SIEM or log service, define alerts for privileged actions (file writes, shell exec, network calls), and retain logs for at least 30 days. Use the built‑in audit hooks of Claude Managed Agents or OpenAI Agents, and supplement with a lightweight watcher (e.g., Cloudflare Workers AI + Durable Objects) to enforce real‑time guardrails.

What does “continuous monitoring” mean for an AI agent that can edit files or run commands?

Unlike a static API call, an agent that can invoke a shell or write to a filesystem can change the environment in ways that are hard to reproduce later. Continuous monitoring captures every action as it happens, stores it in an immutable log, and triggers alerts when a predefined rule is violated. The goal is to detect malicious or accidental behavior before it impacts production.

Which events should I capture?

• Command execution: the exact command string, user‑context, timestamp, and exit code.
• File system writes: path, operation type (create, modify, delete), file hash before/after, and the agent ID.
• Network requests: destination URL, HTTP method, headers, and response status.
• Agent lifecycle: start, stop, restart, and any credential refresh.
• Error traces: stack traces or tool‑specific error messages that may indicate a failed injection attempt.

All events should be emitted as JSON so downstream tools can index and query them efficiently.

How can I emit structured logs from Claude Managed Agents or OpenAI Agents?

Both platforms expose a metadata field that you can populate from your toolchain. For Claude Managed Agents, the official overview describes a log_event helper that forwards a JSON payload to a webhook you control. OpenAI Agents provide a log function in the SDK that can be wired to any HTTP endpoint.

# Example in Python using OpenAI's SDK
import openai, json, requests

def log_event(event):
    requests.post(
        "https://logs.mycompany.com/ingest",
        json=event,
        headers={"Authorization": f"Bearer {API_TOKEN}"}
    )

agent = openai.Agent.create(
    name="file‑editor",
    tools=[...],
    callbacks={"on_tool_use": log_event}
)

Send the payload to a log collector (e.g., Elastic Cloud, Logflare, or a self‑hosted Loki instance) that supports query‑time filters.

What guardrails can I enforce in real time?

1. Command whitelist: only allow a pre‑approved list of binaries (e.g., grep, sed, jq). Reject any other command and raise an alert.
2. File‑path sandbox: restrict writes to a designated directory (e.g., /tmp/agent_workspace). If the agent attempts to write outside, terminate the session.
3. Network egress policy: block calls to external domains that are not on an allow‑list. Use a proxy that can inspect the request and log the destination.
4. Rate limiting: cap the number of shell executions per minute to prevent abuse.

Implement these rules in a lightweight edge worker that intercepts the webhook payload before it reaches your log store. Cloudflare Workers AI can evaluate the JSON and return a 403 response if a rule is violated.

How do I set up alerts for suspicious activity?

Most log platforms let you define a query‑based alert. Example alert for any rm -rf /‑like command:

SELECT * FROM logs
WHERE event.type = "command_execution"
  AND event.command ILIKE "%rm -rf%"
  AND timestamp > now() - interval '5 minutes';

Configure the alert to send a Slack or email notification, and optionally invoke an automated remediation step (e.g., kill the agent process via the platform’s admin API).

How long should I retain logs?

Regulatory guidance (e.g., NIST AI RMF) recommends retaining security‑relevant logs for at least 30 days, with longer periods for incident‑response investigations. Store logs in an immutable bucket (e.g., Cloudflare R2 with versioning) to prevent tampering.

How can a small team implement this without a large budget?

• Use free tiers of log aggregation services (Logflare offers 10 GB/month for free).
• Deploy a single Cloudflare Worker script (< 5 USD/mo) to enforce guardrails.
• Leverage open‑source Loki + Grafana for on‑premise storage if you already have a server.

Even a minimal stack gives you visibility into every privileged action the agent performs.

When should I involve AISecAll?

If you need a custom monitoring pipeline, threat‑modeling assistance, or a managed SIEM integration, AISecAll can design, deploy, and tune the observability stack to match your risk profile.

What are the next steps?

1. Identify the managed‑agent platform (Claude, OpenAI, Replit, etc.).
2. Enable the platform’s built‑in logging hook and point it at a webhook you control.
3. Deploy a lightweight edge worker to enforce the whitelist, sandbox, and rate‑limit rules.
4. Configure alerts for any command, file, or network activity that falls outside the approved list.
5. Set log retention to ≥ 30 days and test the incident‑response playbook.

With these steps, you gain continuous visibility and can react to misbehaving AI agents before they cause damage.