How to Automate Code Review with AI in GitHub Actions

by

How to Automate Code Review with AI in GitHub Actions

Code review is one of the highest-leverage practices in software engineering — and one of the most common bottlenecks. This tutorial shows you how to automate code review with AI in GitHub using Claude and GitHub Actions, so every pull request gets an immediate first-pass review while human reviewers focus on architecture and intent.

This is written for senior engineers who are — reasonably — skeptical of AI review bots. The goal here is a high-signal, low-noise workflow with clear guardrails, not a chatbot spamming your PRs.

The Bottleneck: Manual PR Reviews in Engineering Teams

Every engineering team knows the pattern:

  • A PR sits idle because reviewers are heads-down on their own work.
  • Reviews arrive in fragments — a style nit here, a variable name there — while a real logic bug slips through.
  • Senior engineers spend review time on mechanical checks (missing error handling, unused imports, inconsistent naming) instead of design questions.
  • Context switching costs the reviewer more than the review itself.

None of this means human review is broken. It means human review is being spent on the wrong layer. Mechanical, pattern-based feedback is exactly what a machine should handle; judgment about system design, product intent, and trade-offs is what humans should handle.

How an AI Code Review Workflow Empowers Engineering Teams

A well-designed AI code review workflow for an engineering team does three things:

  1. Immediate first pass. Within a minute or two of opening a PR, the author gets feedback on obvious issues — before a human reviewer ever looks at it. Authors fix the easy stuff early; reviewers see a cleaner diff.
  2. Consistent summaries. An AI pull request summary and feedback comment gives reviewers a plain-language description of what changed and why it might be risky, cutting the “what am I even looking at” phase of review.
  3. Never a gatekeeper. The AI comments; it does not approve, does not block, and does not merge. Keeping the AI advisory-only is the single most important design decision. It sidesteps the hallucination problem: a wrong AI comment costs a human a few seconds of dismissal, not a broken main branch.

Teams adopting AI coding workflows consistently find the value is in shifting human attention up the stack — not replacing it.

Setting Up Claude Pull Request Review Automation

Here’s the full Claude pull request review automation setup, from API key to working Action.

Generating API Keys and Securing Repository Secrets

  1. Create an API key in the Anthropic Console, scoped for this workflow only. Don’t reuse a personal key.
  2. Add it as a repository secret: in GitHub, go to Settings → Secrets and variables → Actions → New repository secret and name it ANTHROPIC_API_KEY.
  3. Lock down permissions. The workflow needs pull-requests: write (to comment) and contents: read (to read the diff) — nothing more. Grant permissions explicitly in the workflow file rather than relying on defaults.
  4. Consider fork PRs carefully. Secrets are not exposed to workflows triggered by fork PRs on pull_request events by default. For open-source repos, keep it that way; run AI review only on branches from your own repo.

Writing the Claude GitHub Action PR Review Setup (YAML examples)

Create .github/workflows/ai-code-review.yml:

[IMAGE: YAML configuration to automate code review with AI GitHub Actions]

name: AI Code Review

on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read
  pull-requests: write

jobs:
  ai-review:
    runs-on: ubuntu-latest
    # Skip drafts and bot-authored PRs to reduce noise
    if: github.event.pull_request.draft == false
    steps:
      - name: Checkout
        uses: actions/checkout@v7
        with:
          fetch-depth: 0

      - name: Get diff
        run: |
          git diff origin/${{ github.base_ref }}...HEAD \
            -- . ':(exclude)package-lock.json' ':(exclude)*.min.js' \
            > pr.diff
          head -c 150000 pr.diff > pr_truncated.diff

      - name: Review with Claude
        env:
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          python .github/scripts/ai_review.py \
            --diff pr_truncated.diff \
            --pr-number ${{ github.event.pull_request.number }} \
            --repo ${{ github.repository }}

The review script calls the Claude API with a constrained prompt and posts a single consolidated comment:

# .github/scripts/ai_review.py (core logic)
import anthropic

REVIEW_PROMPT = """You are a senior code reviewer. Review this diff.

Rules:
- Report ONLY: probable bugs, security issues, missing error handling,
  and breaking API changes.
- Do NOT comment on style, formatting, or naming.
- If you are not confident an issue is real, do not report it.
- If the diff looks fine, say exactly: "No significant issues found."

Output format:
1. **Summary** (2-3 sentences: what this PR does)
2. **Risk level** (low / medium / high, one line of justification)
3. **Findings** (bulleted; file and line reference for each)

Diff:
{diff}"""

client = anthropic.Anthropic()
msg = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1500,
    messages=[{"role": "user",
               "content": REVIEW_PROMPT.format(diff=diff_text)}],
)

Key design choices worth copying even if you write your own script:

  • One consolidated comment, not line-by-line spam. Update the same comment on new pushes instead of posting duplicates.
  • Exclude lockfiles and generated code from the diff — they burn tokens and produce useless findings.
  • Truncate large diffs and say so in the comment, rather than silently reviewing half a PR.

For a deeper walkthrough of the review-script side, see our guide on how to use Claude to review pull requests.

[IMAGE: GitHub pull request screen showing AI code review workflow]

Customizing AI Pull Request Summary and Feedback

The default prompt above is deliberately conservative. Tune it to your team:

  • Encode your standards. Add house rules to the prompt: “All public functions must have error handling,” “Database queries must be parameterized,” “New endpoints require auth middleware.” The AI now checks your checklist, consistently, on every PR.
  • Set a confidence bar. The instruction “if you are not confident, do not report it” measurably reduces false positives. High precision beats high recall for review bots — the first week of noisy comments is when engineers decide to ignore the bot forever.
  • Route by path. Use different prompts for infra/, api/, and frontend/ directories. Infrastructure changes deserve questions about rollback and blast radius; frontend changes don’t.
  • Label risk. Have the AI assign a risk level, then use it in automation — e.g., auto-request a second reviewer on “high” — the same way you’d automate repetitive DevOps tasks with any other signal.

Security Best Practices for Exposing Code to LLMs

Sending source code to an LLM API is a real security decision. Handle it explicitly:

  • Review your API provider’s data-use terms and confirm your tier’s retention and training policies meet your company’s requirements before enabling the workflow.
  • Never send secrets. Add a pre-step that scans the diff with a secret scanner (e.g., gitleaks) and aborts the AI review if anything is detected. A leaked credential in a diff is a leaked credential, full stop.
  • Exclude sensitive paths. Keep crypto implementations, auth internals, or proprietary algorithms out of the diff via pathspec excludes if policy requires it.
  • Scope the key, rotate the key. One key per workflow, rotated on a schedule, revoked instantly if the repo’s trust model changes.
  • Keep humans in the merge path. Branch protection rules should still require human approval. The AI is a reviewer, never an approver.

For teams whose policies rule out external APIs entirely, the same workflow pattern works with a self-hosted model endpoint — the YAML barely changes; only the API URL does.

FAQ

Will the AI reviewer block my pull requests?

Not in this setup. The workflow only posts comments — it never approves, rejects, or merges. Branch protection and human approval remain exactly as they were. This advisory-only design is deliberate and recommended.

How do I stop the AI from posting noisy or wrong comments?

Constrain the prompt: restrict findings to bugs, security, and error handling; forbid style commentary; and instruct the model to skip anything it isn’t confident about. Post one consolidated comment per PR and update it on new pushes.

Is it safe to send our code to the Claude API?

That depends on your provider agreement and internal policy. Review data-retention terms, exclude sensitive paths, scan diffs for secrets before sending, and scope API keys tightly. If external APIs are off the table, run the same workflow against a self-hosted model.

What does the AI review cost per pull request?

Cost scales with diff size and model choice. Excluding lockfiles and generated files, truncating oversized diffs, and using a mid-tier model keeps per-PR cost low; measure it on your own traffic for a real number rather than trusting generic estimates.

Does this work for pull requests from forks?

By default, GitHub does not expose secrets to workflows triggered by fork PRs — which is the safe behavior. Run AI review on same-repo branches, and treat fork PRs with standard human-only review.

Leave a Comment