Use Runtime History with Lightrun MCP🔗

Runtime History enables an AI agent to investigate application behavior using snapshot data that Lightrun already captured. This allows an investigation to begin immediately, even when an incident has ended or the relevant code path is difficult to reproduce.

The AI agent decides which evidence it needs. It uses Runtime History when existing captures can support an accurate answer and collects fresh runtime data when the historical evidence is unavailable, outdated, or incomplete.

Why use Runtime History?🔗

Runtime investigations often begin after the behavior being investigated has already occurred. Without historical evidence, the AI agent may need to create a new snapshot and wait for the same behavior to happen again.

Runtime History allows the AI agent to:

• Investigate incidents after they have ended.
• Analyze intermittent behavior without waiting for it to recur.
• Reuse captured values and stack traces from existing snapshot hits.
• Narrow evidence by repository, commit, source location, runtime source, time range, or captured value.
• Reduce unnecessary on-demand actions when sufficient evidence already exists.
• Combine historical evidence with fresh runtime data when further validation is required.

How Runtime History works🔗

When Runtime History is enabled, Lightrun sends captured snapshot hits to the Runtime Collector. The Runtime Collector stores the hit metadata, captured values, and stack traces in a ClickHouse database until their retention period expires.

An authorized MCP client can then:

1. Discover which source files or functions have captured history.
2. Search individual historical hits.
3. Retrieve the captured values and stack trace for a selected hit.
4. Continue with a live Lightrun action when historical evidence is unavailable or incomplete.

Runtime History stores snapshot hits that Lightrun captured while the feature was enabled. It is not a continuous code-coverage or tracing service. An empty result does not prove that a code path did not execute.

Availability by deployment type🔗

Prerequisites🔗

Before using Runtime History, ensure that:

• Your Lightrun deployment is version 1.97 or later.
• A Company Admin has enabled both MCP access and Collector – Runtime History in Service Configuration.
• Your AI agent is connected to Lightrun MCP. See the Lightrun MCP quickstart.
• Your role or API key permits access to the relevant agent pools.
• Lightrun snapshots have captured hits while Runtime History was enabled.
• For an On-Premises deployment, Runtime Collector and its dependencies are running.

Configure retention🔗

Runtime History is enabled by default and retains captured data for 30 days. A Company Admin can select a retention period of 30, 60, or 90 days.

To configure Runtime History:

1. Log in to the Lightrun Management Portal.
2. Click Settings in the upper-right corner.
3. Under Compliance, select Service configuration.
4. Find Collector – Runtime History.
5. Turn Collector – Runtime History On or Off.
6. When Collector – Runtime History is On, select 30 days, 60 days, or 90 days under Retain data for.
7. Save the service configuration.

Turning Collector – Runtime History Off stops new snapshot hits from being stored and removes the Runtime History tools from the organization's MCP tool list. Previously stored history remains subject to its existing expiration time.

When the AI agent uses Runtime History🔗

The user does not need to select Runtime History or choose individual MCP tools. The user asks a question in natural language, and the AI agent selects the appropriate investigation path.

The AI agent uses Runtime History when:

• The question concerns a past incident or time range.
• The behavior is intermittent or difficult to reproduce.
• Relevant snapshot hits may already exist.
• Existing captures may answer the question without collecting more data from the running application.

The AI agent uses on-demand MCP tools when:

• The question requires the application's current state.
• No relevant historical captures exist.
• The available captures are too old or do not contain the required values.
• Historical evidence suggests a hypothesis that requires fresh validation.

An investigation can use both sources. Runtime History provides the starting evidence, while on-demand tools fill any remaining gaps.

How an AI agent investigates🔗

Runtime History provides the following read-only MCP tools:

The AI agent conducts an investigation as follows:

1. Understand the question. The AI agent identifies the relevant service, environment, repository, source location, commit, and incident time range.
2. Check available history. It uses query_runtime_history_coverage to determine which files or functions contain relevant captures.
3. Inspect relevant hits. It uses query_runtime_history_hits to narrow the results and retrieve only the captured values, variable information, and stack traces needed for the investigation. It can use get_runtime_history_hit to retrieve a known hit again.
4. Evaluate the evidence. It determines whether the historical captures are relevant, sufficiently recent, and complete enough to support a conclusion.
5. Collect fresh evidence when necessary. If history is missing or insufficient, it uses an on-demand tool such as snapshot_create to inspect the running application.
6. Combine and explain the findings. It presents the historical and live evidence, explains its conclusion, and identifies any remaining uncertainty.

The AI agent reuses the same correlationKey across Runtime History calls and on-demand actions, allowing Lightrun to associate the evidence with one investigation.

For complete parameters, limits, and examples, see Supported Lightrun MCP tools.

Example: Investigate an overnight checkout failure🔗

A user asks:

Investigate why checkout requests failed overnight in production and explain the runtime evidence supporting your conclusion.

The AI agent:

1. Determines the production runtime source and incident time range.
2. Checks whether Runtime History contains captures for the checkout service.
3. Searches the matching hits for the affected commit, file, or function.
4. Examines relevant captured values and stack traces.
5. Determines whether the captures explain the failure.
6. Creates a live snapshot only if additional or current evidence is required.
7. Reports the conclusion and distinguishes historical evidence from newly collected evidence.

The user does not need to know which MCP tools perform these steps.

Security and data handling🔗

Runtime History uses the same authorization boundaries as other Lightrun MCP capabilities:

• OAuth and personal access token clients use the current user's RBAC permissions and Agent Pool access.
• API-key clients use the key's scopes and Agent Pool Access configuration.
• Queries return data only from agent pools that the caller can access.
• Unknown and unauthorized hit IDs return the same not-found response.
• PII redaction applies before captured values are returned. A redacted value is identified by the redacted outcome and does not include its text.

For more information, see Lightrun MCP security and privacy.

Deploy Runtime Collector on-premises🔗

For Runtime History in an On-Premises deployment, use Lightrun Helm chart 3.56.0 or later. Runtime Collector is disabled by default. Enable it in your values.yaml:

deployments:
  runtime_collector:
    enabled: true

Runtime Collector requires:

• RabbitMQ for the snapshot-hit event queue.
• A ClickHouse database for Runtime History storage.
• A shared gRPC secret for communication between the Lightrun backend and Runtime Collector.
• Matching Runtime Collector server and database-migration image versions.

For the complete Helm values reference, see Runtime Collector.

Configure secrets🔗

When the chart manages secrets, provide the Runtime Collector gRPC secret and local ClickHouse credentials:

secrets:
  runtime_collector:
    grpc_secret: "<strong-shared-secret>"
  clickhouse:
    user: "<clickhouse-user>"
    password: "<clickhouse-password>"

If you manage all Lightrun secrets outside the chart, disable chart-managed secrets and provide the pre-provisioned secret names:

general:
  deploy_secrets:
    enabled: false
    existing_secrets:
      backend: "<backend-secret-name>"
      keycloak: "<keycloak-secret-name>"
      clickhouse: "<clickhouse-secret-name>"
      runtime_collector: "<runtime-collector-grpc-secret-name>"

This setting disables chart management for every Lightrun secret, so pre-create all required secrets, including the backend and Keycloak secrets. The ClickHouse secret must contain CLICKHOUSE_USERNAME and CLICKHOUSE_PASSWORD. The Runtime Collector secret must contain RUNTIME_COLLECTOR_GRPC_SECRET, which is shared by the backend and Runtime Collector. For the complete requirements, see Manage Helm chart secrets.

Choose and size the ClickHouse database🔗

The Helm chart defaults are lower than the recommended resources. Set the following values explicitly:

The following example sets the recommended resources for Runtime Collector and a local ClickHouse:

deployments:
  runtime_collector:
    enabled: true
    server:
      resources:
        requests:
          cpu: "1"
          memory: 4Gi
        limits:
          cpu: "1"
          memory: 4Gi
    clickhouse:
      local:
        enabled: true
        resources:
          requests:
            cpu: "2"
            memory: 8Gi
          limits:
            cpu: "2"
            memory: 8Gi

Lightrun Support can help plan ClickHouse disk capacity for your selected data-retention period.

Option 1: External ClickHouse (recommended)🔗

For production deployments, we recommend using an external ClickHouse service. Supported options include:

• Managed ClickHouse Cloud.
• A ClickHouse Operator.
• A ClickHouse Helm chart.
• Another managed ClickHouse solution.

The external ClickHouse deployment must meet the following requirements:

• ClickHouse version 25.3 or later.
• The host must be reachable from the Runtime Collector pod on both the HTTP or HTTPS port and the native port.
• The credentials must be able to create the runtime_collector database, unless an administrator creates it beforehand.
• For high availability, use a replicated ClickHouse deployment and set external.cluster to the cluster name.
• Provide credentials inline or through external.existingSecret. The referenced secret must contain CLICKHOUSE_USERNAME and CLICKHOUSE_PASSWORD.

For example, connect to ClickHouse Cloud:

deployments:
  runtime_collector:
    clickhouse:
      local:
        enabled: false
      external:
        host: "abc123.us-east-1.aws.clickhouse.cloud"
        httpPort: 8443
        nativePort: 9440
        tls: true
        existingSecret: "clickhouse-credentials"

Option 2: Local ClickHouse (development or smaller installations)🔗

The chart can deploy ClickHouse inside the Kubernetes cluster. Local ClickHouse has the following considerations:

• It runs as a single replica with the Recreate rollout strategy. Clustered ClickHouse is supported only for external instances.
• Persistent storage is recommended. Set persistence.enabled to true and provide a pre-created, customer-managed ReadWriteOnce persistent volume claim in existingClaim.
• The default 10 GiB EmptyDir is for testing only. Runtime History data is lost when the pod is replaced.
• You are responsible for backing up and restoring the persistent storage; the chart does not back up local ClickHouse data.
• Credentials come from secrets.clickhouse, or from the Kubernetes secret set in general.deploy_secrets.existing_secrets.clickhouse.

To configure persistent storage:

deployments:
  runtime_collector:
    clickhouse:
      local:
        enabled: true
        persistence:
          enabled: true
          existingClaim: "clickhouse-data"

Configure RabbitMQ🔗

Keep general.mq.enabled set to true. When the Helm chart deploys RabbitMQ locally, it creates the Runtime Collector exchange, queue, and dead-letter queue.

When you use an external RabbitMQ deployment, create the snapshot-event exchange and queues before starting Runtime Collector. The RabbitMQ message time-to-live settings buffer events awaiting ingestion; they do not control the 30-, 60-, or 90-day Runtime History retention period.

Unless you override general.mq.snapshot_events, provision the following durable objects in the / virtual host:

Configure the main queue to dead-letter rejected messages to runtime-collector-snapshot-events.dlq.

Increase deployments.runtime_collector.server.replicas to run multiple Runtime Collector pods. The pods consume snapshot events concurrently from the shared queue.

Configure internal TLS🔗

When internal TLS is enabled, provide certificates for the backend, Runtime Collector, and local ClickHouse. Both sides of the backend-to-collector gRPC connection must trust the issuing certificate authority.

Troubleshooting🔗

Runtime History tools are not listed🔗

Verify that:

• The deployment is running Lightrun 1.97 or later.
• MCP access is On.
• Collector – Runtime History is On.
• For an On-Premises deployment, Runtime Collector is enabled and healthy. SaaS and Single-Tenant deployments use the managed Runtime Collector.
• The MCP client reconnected after the configuration changed so that it could reload the tool list.

A query returns no history🔗

An empty response means no stored hit matched the query and the caller's authorized agent-pool scope. It does not prove that the code did not execute.

Check that:

• A snapshot captured the relevant code while Runtime History was enabled.
• The selected repository, commit, file, function, runtime source, and time range are correct.
• The captured hit has not passed its retention period.
• The caller can access the matching agent pool.

If no suitable history exists, use an on-demand snapshot tool to collect fresh data.

On-Premises Runtime Collector does not start🔗

Check the Runtime Collector init containers and verify that:

• RabbitMQ is reachable and the snapshot-event queue exists.
• ClickHouse is reachable and the configured credentials are valid.
• The Runtime Collector and migration images use the same version.
• The backend and Runtime Collector use the same gRPC shared secret.
• TLS certificates are trusted in both directions.