ReadMe Realist automatically detects structural code deltas (CLI flags, environment variables, dependencies, routes) during Pull Requests and alerts developers before outdated docs break production.
The Documentation Drift Dilemma
Software evolves rapidly during Pull Requests. Developers add environment variables, update CLI parameters, or change API routes—but updating documentation is easily forgotten.
REDIS_URL env variable added to code, but absent in README.
An intelligent GitHub App that intercepts PR webhooks, parses structural diffs, extracts key signals, and evaluates changes against repository markdown using LLMs.
End-to-End System Dataflow
Live PR Drift Evaluator Simulator
Simulate incoming code deltas and observe how ReadMe Realist parses signals and creates GitHub feedback.
Data Transformation Lifecycle
Detailed breakdown of the 9 execution stages from GitHub webhook triggers to PR comment upserts.
| Step ID | Source Node | Target Node | Data Contract / Payload | Transformation & Logic |
|---|---|---|---|---|
| 01 | GitHub PR Event | Webhook Receiver | JSON + X-Hub-Signature-256 | Delivers webhook delivery with HMAC-SHA256 signature headers. |
| 02 | Webhook Receiver | Signature Validator | Raw Bytes + Secret Key | Validates cryptographic signature before parsing payload to prevent tampering. |
| 03 | Webhook Receiver | Background Worker | PullRequestContext Object | Sanitizes payload into domain model and enqueues review task asynchronously. |
| 04 | Pipeline Orchestrator | GitHub REST API | Authenticated JWT | Creates initial GitHub Check Run in in_progress state and downloads PR diff. |
| 05 | Diff Downloader | Code Delta Parser | Unified Diff String | Filters out binary/lockfile noise and extracts structural signals (CLI, env, deps, routes). |
| 06 | Delta Parser | Doc Fetcher / GitHub | File Globs (README.md, docs/**) | Scans repo for target documentation files matching configured glob patterns. |
| 07 | Pipeline Orchestrator | LLM Evaluator | DiffAnalysis + DocBundle | Formats context and prompt instructions requesting structured DriftVerdict JSON. |
| 08 | LLM Backend | Feedback Orchestrator | DriftVerdict JSON | Parses model response into verdict structure with proposed markdown patches. |
| 09 | Feedback Orchestrator | GitHub PR & Checks | Markdown Comment + Check Run | Upserts single persistent PR review comment (via marker tag) and marks check completion. |
Guardrails & Failure Resilience
PRs with no code changes, formatting/whitespace-only updates, or doc-only edits immediately skip LLM inference, reducing evaluation latency and saving API token costs.
If external LLM services experience rate limits or network timeout errors, the pipeline gracefully posts a neutral check run without blocking developer PR merges.
Feedback comments are tagged with an invisible HTML marker (<!-- readme-realist:v1 -->) and upserted in-place to prevent comment spamming across repeated pushes.
Deployment & Configuration Guide
Follow these steps to deploy ReadMe Realist locally, via Docker, or directly as a GitHub App.
https://your-domain.com/webhooks/github or Smee URL)../secrets/github-app.private-key.pem.