Runtime-Aware PR Review Skill🔗

The Runtime-Aware PR Review Skill helps an AI assistant review pull requests with evidence from live application behavior. It guides the assistant to compare the pull request against the deployed code, check whether the changed areas are observable in runtime, collect production samples when needed, and simulate the proposed change against captured inputs.

The skill is available in the lightrun-ai repository as lightrun-runtime-aware-pr-review.

Before using this skill, install the shared Lightrun AI Skills package and complete the requirements in Lightrun AI Skills.

When to use this skill🔗

Use the Runtime-Aware PR Review Skill when you want an AI assistant to review a pull request with runtime evidence instead of relying only on static code analysis.

This skill is useful when:

• A pull request changes code that already runs in production or staging.
• You want to understand how the changed code behaves with real inputs.
• You need to verify whether a suspected regression can occur in live traffic.
• You want review findings grounded in deployed behavior, runtime samples, and patch simulation.
• You want to add runtime-aware review as an automated pull request check for every code change.

How to use this skill🔗

Start a new AI assistant chat and ask the assistant to use the Runtime-Aware PR Review Skill for the pull request you want to review.

Use a prompt like this:

Use the Lightrun Runtime-Aware PR Review Skill to review this pull request.

Pull request:
https://github.com/example-org/example-service/pull/123

Target service:
checkout-service in production.

What I want to verify:
The PR changes discount calculation logic. Please compare it against the deployed version, inspect runtime behavior for the affected code path, and identify any behavior changes that could affect production traffic.

The assistant should identify the deployed baseline, compare it with the pull request, determine which changed areas can be verified at runtime, and use Lightrun MCP to collect production evidence where needed.

Use this skill in PR review automation🔗

Engineering managers can use the Runtime-Aware PR Review Skill as part of their pull request automation. Instead of waiting for a developer to ask for runtime verification manually, the automation can invoke the skill whenever a pull request is opened, updated, or marked ready for review.

This adds a runtime-aware review check to the existing pull request process. The AI assistant reviews the code change, connects it to the deployed service where possible, collects runtime context through Lightrun MCP, and publishes the result back to the pull request as an additional review signal.

For automated PR checks, a Lightrun administrator must generate an API key in the Lightrun Management Portal. The automation runner uses this API key to authenticate to the Lightrun server and access the runtime sources needed for review. Store the API key as a secret in the automation system, such as a CI secret, repository secret, or workflow secret, and scope access to the services and environments the check should review.

For more information, see API Keys.

A typical automation connects:

• The source control system, such as GitHub or GitLab.
• The AI assistant or automation runner that can execute the skill.
• Lightrun MCP, authenticated with the Lightrun API key and connected to the relevant production or staging runtime sources.
• The pull request comment, check, or status destination where the review result should be published.

The following flow shows how the automation handles both immediate runtime findings and findings that arrive later:

Use a prompt like this in the automation:

For every ready-for-review pull request, run the Lightrun Runtime-Aware PR Review Skill.

Review the pull request against the deployed service when runtime context is available.
Use Lightrun MCP to collect runtime evidence for changed code paths.

If no live traffic triggers the relevant code path during the initial review, keep the runtime action active and poll for new results at the configured interval.
When new runtime findings are available:
- If the pull request is still open, update the pull request review or check with the new findings.
- If the pull request is already merged, add the findings as a pull request comment so reviewers still have the runtime context.

Return:
- Runtime-backed findings.
- Areas that could not be verified with runtime evidence.
- Residual risk for reviewers.
- Any runtime findings collected after the initial review.
- A concise recommendation for whether the pull request needs follow-up before merge.

This helps teams make runtime context a standard part of code review. Reviewers still make the final merge decision, but every pull request can include an additional signal based on how the changed code relates to live application behavior. For low-traffic code paths, the automation can continue collecting runtime evidence after the initial review and publish new findings when they become available.

What the skill does🔗

The skill gives the assistant a structured workflow for reviewing pull requests with runtime evidence:

• It treats the deployed production code as the baseline for review.
• It compares the deployed commit with the pull request head.
• It identifies changed areas that can be verified with runtime evidence.
• It checks historical runtime coverage before creating new runtime actions.
• It uses live snapshots only for areas that are not already covered.
• It simulates the pull request behavior against captured production inputs.
• It produces review findings that separate runtime evidence from inferred conclusions.

This approach helps the assistant focus review attention on behavior that is observable in live systems and avoids presenting static assumptions as production facts.

Expected review flow🔗

During a successful runtime-aware pull request review, the assistant should:

1. Identify the deployed commit for the target service.
2. Compare the deployed baseline with the pull request head.
3. Identify the changed code areas that need runtime verification.
4. Determine whether each changed area is verifiable with Lightrun MCP.
5. Search for existing runtime profile or historical hit evidence.
6. Create a focused sampling plan only for changed areas that still need coverage.
7. Collect live snapshot samples from the selected runtime sources.
8. Simulate the pull request behavior against captured production inputs.
9. Produce a final review with runtime-backed findings, residual risks, and any areas that could not be verified.

Runtime evidence and review findings🔗

Runtime-aware review works best when the assistant can connect the pull request diff to a deployed code path that receives traffic. For each finding, the assistant should explain:

• The changed code path being reviewed.
• The deployed baseline behavior.
• The production inputs or runtime values observed.
• The simulated pull request behavior.
• The resulting review verdict.
• Any remaining uncertainty or missing coverage.

When runtime verification is incomplete, the assistant should say so directly and continue with a scenario-driven review for the areas that could not be verified.

Troubleshooting🔗

The assistant cannot find Lightrun MCP tools🔗

Confirm that Lightrun MCP is installed, enabled, and authenticated in your AI client. Then ask the assistant to list the available Lightrun MCP tools.

For setup instructions, see the MCP quickstart guide.

The automated review has limited runtime context🔗

Runtime-aware review depends on the runtime sources available to Lightrun MCP and the code paths that receive traffic during the review. When runtime evidence is not available for part of a pull request, the assistant should state that limitation clearly and continue with scenario-driven review for that area.

Related topics🔗

• Lightrun AI Skills
• MCP quickstart guide
• Supported Lightrun MCP tools