Audit MCP server effects during an agent run

The short version: an MCP result is not a system-effect ledger

MCP is the semantic boundary. It can tell the host that a tools/call request named a particular tool, supplied particular arguments, and returned a particular result. The operating system sees a different layer: the server executable, its standard streams or sockets, child processes, file operations, and network activity. Neither record automatically reconstructs the other.

That is the useful audit model for AgentSight. Start with the protocol record because it has better semantic labels. Add independent process, file, network, and stdio/TLS observation when the question crosses into what the implementation actually did. A mismatch is a reason to investigate, not an automatic vulnerability verdict.

Freeze the protocol revision before you compare anything

The current MCP specification is 2026-07-28. It moved the core protocol to stateless, self-contained requests and removed the connection-scoped initialize/initialized handshake and protocol-level session used by earlier revisions. Request metadata now travels with each request, while Streamable HTTP mirrors selected fields into headers for routing and inspection.

This matters for audits because a trace from a legacy client and a trace from a current client can have different message sequences even when they call the same logical tool. Record the MCP revision, transport, client implementation/version, server build or commit, working directory, relevant authorization mode, and the exact test task before comparing observed effects.

The shipped AgentSight fixture is a capture baseline, not current MCP conformance

AgentSight v1.0.25 still ships a deliberately small Python fixture under docs/experiment/mcp-test/. It uses only the standard library and exposes three predictable tools: echo, sum_numbers, and read_fixture. The client exercises both stdio and HTTP modes and the file-read tool returns a fixed marker string from fixture_note.txt.

The fixture client explicitly advertises protocol version 2025-03-26 and performs the legacy initialize then notifications/initialized exchange. The current 2026-07-28 specification removed that handshake. Use this fixture to prove that your capture path can see known traffic and effects; do not use it to claim that a server is conformant with the latest MCP revision.

Terminal

$ python3 docs/experiment/mcp-test/test_mcp_cli.py --transport stdio
$ python3 docs/experiment/mcp-test/test_mcp_server.py --transport http --host 127.0.0.1 --port 8765
$ python3 docs/experiment/mcp-test/test_mcp_cli.py --transport http --url http://127.0.0.1:8765/messages

These commands use the actual v1.0.25 repository path. Older fixture prose still contains a shorter docs/mcp-test/ path, but the files in the pinned release live under docs/experiment/mcp-test/.

Know exactly what the deterministic fixture should do

Fixture step	Protocol-level observation	Expected implementation effect
`tools/list`	Returns `echo`, `sum_numbers`, and `read_fixture`.	No external service is required by the fixture implementation.
`echo`	Returns a stable `echo:<transport>-hello` text result.	String handling occurs inside the Python server process.
`sum_numbers`	The client sends 1, 2, 3, and 4.5; the server returns `sum:10.5`.	Numeric conversion and summation occur inside the server process.
`read_fixture`	Returns the fixed AgentSight fixture payload as text.	The server reads `fixture_note.txt` from its own fixture directory.

This gives the audit a known positive control. If the protocol result is present but the expected file read is outside your configured observation scope, diagnose capture first. If a third-party server shows additional descendants or destinations, you then have a concrete difference to explain rather than a vague “MCP did something” alert.

stdio is plaintext, but the capture unit is a syscall, not an MCP message

In the current MCP stdio binding, the client launches the server as a subprocess, writes newline-delimited JSON-RPC messages to the server's stdin, and reads server messages from stdout. AgentSight's standalone stdiocap path observes Linux read and write syscalls and can scope capture to file descriptors 0, 1, and 2 and to a target process family.

The v1.0.25 BPF program records PID, TID, UID, fd, direction, returned length, command name, and a bounded copy of the user buffer. The event buffer is capped at 8192 bytes. That means a stdio event is a syscall chunk, not proof that one event equals one complete JSON-RPC line. Large messages can be truncated, and application buffering can divide protocol messages across writes or reads. Reconstruct protocol semantics above the raw capture layer before making tool-level claims.

Terminal

$ sudo ./bpf/stdiocap -p <mcp-server-pid>

Streamable HTTP has a different observation surface in MCP 2026-07-28

Current Streamable HTTP uses a single MCP endpoint and a new POST for every request or notification. The July 2026 revision removed the old GET stream endpoint and protocol-level sessions. Modern requests carry MCP-Protocol-Version and mirror the JSON-RPC method into Mcp-Method; requests such as tools/call also carry Mcp-Name. The request body remains the source of truth.

The AgentSight fixture's HTTP server is intentionally simpler and accepts /messages or /mcp with ordinary JSON bodies. It is useful as a local network baseline, but it does not implement all current Streamable HTTP requirements. When auditing a current remote server, preserve the protocol version and relevant routing headers alongside the JSON-RPC body, destination, TLS boundary, and process or service identity.

Compare three layers instead of flattening everything into one finding

Layer	Good examples	What it cannot prove alone
Protocol fact	Tool name, arguments, protocol revision, returned result, error state.	Every child process, file operation, or network effect produced by the implementation.
Observed system fact	Server process, descendants, path activity, destinations, stdio or supported TLS traffic.	Why the model selected the tool or whether an observed effect was authorized.
Reviewer inference	“This destination was unnecessary” or “this helper explains the returned result.”	Anything without a stated reasoning path and reproducible source facts.

Keeping these layers separate makes the audit easier to challenge. A reviewer can disagree with an inference without disputing that a process started or a tool returned a particular value.

Tool descriptions and annotations are claims, not independent verification

The current MCP tools specification treats tools as model-controlled capabilities and recommends clear user visibility and confirmation around invocation. It also says tool annotations should be considered untrusted unless they come from a trusted server. For an audit, record the advertised schema and description, but compare them with the bounded task and observed implementation behavior rather than treating metadata as an allowlist.

The same rule applies in reverse: an observed cache file, helper process, or network connection is not automatically malicious because the tool description did not enumerate it. Classify expected project paths, runtime/configuration paths, temporary/cache activity, expected service endpoints, localhost, and unexplained external effects separately.

Repeat surprising effects with one variable changed at a time

Re-run the same call before escalating a one-off observation. If the same child process, path, or destination appears at the same tool phase, the attribution is stronger. Then change one input, permission, transport, or server configuration and repeat. This is especially useful on developer machines where language servers, package managers, background Git operations, and local services can otherwise contaminate a broad trace.

Record positive observations more strongly than absences. “This process opened this path during the bounded run” is a direct observation. “The server never accessed any other path” requires confidence in process-family scope, probe coverage, buffering/truncation behavior, and the exact runtime path used by the server.

Plaintext MCP capture is sensitive by construction

stdio JSON-RPC can contain tool arguments, returned text, file contents, resource identifiers, credentials passed incorrectly as parameters, and application-specific data. HTTP/TLS capture can expose the same material once observed at a plaintext boundary. Keep raw captures local, redact before sharing, and preserve only the structure needed for the review question.

The 8192-byte stdiocap event bound is also a correctness limit, not only a storage detail. A successful event proves that a syscall payload was observed; it does not guarantee a complete large MCP message. If the finding depends on exact content beyond the captured prefix, use the native protocol log or another bounded source to verify it.

Do not confuse the test fixture with an AgentSight MCP product server

The current AgentSight repository contains a separate MCP server design document, but that document explicitly marks the production MCP server as a design target. The runnable Python fixture is for testing MCP flows; it is not a shipped production AgentSight MCP service. This distinction matters when reviewing architecture diagrams or old implementation notes: audit what exists in the pinned release, not what a design document proposes.

The same discipline applies to a third-party server. Pin the executable, container image, package version, or source commit you actually exercised. A server with the same tool name can change implementation behavior across releases even when the MCP-facing schema stays stable.

A publishable MCP audit should include the method and the uncertainty

Record the server and client versions, MCP revision, transport, task, authorization boundary, AgentSight version, capture commands, relevant process/file/network findings, and any truncation or unsupported-runtime limits. Include the protocol facts that justify each system-side comparison and list unresolved questions separately from confirmed observations.

That produces a useful artifact even when nothing suspicious appears. The result is a scoped compatibility and behavior statement for one server build and one bounded capability, not a certificate that every tool and configuration of the server is safe.