I Built an AI That Turns GitHub Issues Into Pull Requests — No Local Setup Required
What if fixing a bug was as simple as pasting an issue link and clicking one button?

Search for a command to run...
What if fixing a bug was as simple as pasting an issue link and clicking one button?

No comments yet. Be the first to comment.
Building a Retrieval-Augmented Generation (RAG) system that performs well in real-world scenarios is rarely a straight path. My initial implementation started with a standard pipeline, but the evaluation results were not satisfactory. This blog walks through the iterative improvements I made—from data preparation to advanced retrieval and chunking strategies—that eventually led to a significantly better-performing system.

Diving beneath the sleek chat interfaces to find out why LLMs fail at basic counting, why 'answer only' breaks them, and the science of RLHF

Every developer knows the drill. A bug gets filed. You clone the repo (if you haven't already), pull the latest changes, spend 10-15 minutes just understanding where the problem lives, write a fix, write tests, run them, iterate, and finally open a PR. For a small-to-medium issue, this routinely eats 45 minutes to an hour — and that's before code review even starts.
Multiply that across a backlog of 50 "good first issue" tickets and you're looking at entire engineering-days spent on work that's often mechanical: locate → understand → patch → verify → submit.
I wanted to see how much of that loop could be automated — not with a single LLM call and a prayer, but with a system that actually mirrors how an experienced engineer works through an issue.
💡 Prefer hands-on testing? Scroll to the bottom of this post for an interactive live demo and repository links!
resolvo takes two inputs — a GitHub issue and a repository URL — and returns a pull request with passing tests. No local clone. No manual setup. Everything runs remotely, and you get an SSE-streamed view of progress in real time.
The measured result: turnaround time on lightweight-to-medium issues drops by roughly 85%.
Zero local footprint — the entire fix-and-verify loop happens in a remote sandbox. Your machine never touches the repo.
Smart triage — not every issue needs the same amount of machinery. The system classifies each issue by complexity and routes it down a lighter or heavier path accordingly, so simple fixes don't pay the cost of an exhaustive analysis.
Real understanding, not pattern-matching — before writing a single line of code, the system builds an actual map of the codebase: which functions call which, which files depend on which, and where the relevant logic actually lives. This is what separates a fix that compiles from a fix that's correct.
Tests are generated and run, not assumed — every fix is verified in an isolated sandbox before it ever reaches a PR. If tests fail, the system iterates on its own fix.
Built-in code review — a review pass checks the diff for quality issues before the PR goes out, and can request changes just like a human reviewer would, triggering another round of fixes.
Human-in-the-loop where it matters — for anything flagged as high-risk or high-complexity, the system stops and hands off for human review rather than merging blind.
Live progress, not a black box — you see each stage as it happens, not just a final "done."
One of the earliest lessons: treating a one-line typo fix and a cross-module refactor with the same pipeline is wasteful and slow. The system makes an early, fast classification pass and picks one of three paths — a fast track for trivial changes, a standard path for typical fixes with full test coverage, and a critical path for anything that always requires a human to sign off before merging. This routing decision alone is a major contributor to the speed gains on lightweight and medium-weight issues — they simply skip stages that heavier issues need.
Rather than asking a model to guess at a fix from a raw diff view, the system first constructs a structural understanding of the target repository — how files relate to each other, what depends on what. This map is what the planning stage draws from when deciding exactly which files need to change and why, before any code is written.
Locating the actual file(s) relevant to a fix is often the hardest part of debugging — harder than writing the fix itself. Rather than relying on a single retrieval strategy, the planning stage combines several independent signals (keyword-based search, semantic re-ranking, symbol name matching, and dependency-aware expansion) and merges them into a single ranked shortlist. Diversity of signal beats depth of any one signal.
A generated diff isn't trusted until it's been tested in a live, isolated environment. This closes the loop that a lot of "AI writes code" demos skip — a fix that looks plausible but doesn't actually pass tests is worse than no fix at all.
Before a PR reaches a human, it goes through an automated review pass that can reject the change and send it back for another iteration — with the feedback carried forward so the next attempt actually addresses what was flagged, rather than starting from scratch.
The 85% turnaround improvement isn't evenly distributed — it's heavily concentrated in the issue categories that make up the bulk of most backlogs:
Lightweight issues (typos, small logic bugs, minor config changes) skip the heaviest analysis stages entirely and move through a fast, lightly-reviewed path — these go from "sit in the backlog for days" to "resolved in minutes."
Medium issues (a bug that touches 2-3 files, a small feature with defined scope) benefit most from the automated retrieval-and-planning stage — the part of the work that normally costs a human the most thinking time before they write a single line of code.
Critical/complex issues still get a human in the loop by design — this system isn't trying to replace judgment on high-stakes changes, just remove the grunt work leading up to that judgment call.
The net effect: engineers spend their time on the 10-20% of issues that genuinely need human judgment, while the mechanical majority resolve themselves.
I'm continuing to refine the classification thresholds and expanding language support beyond the current scope. If there's interest, I'll follow up with a deeper post on the evaluation methodology — how "success" is measured across issue types, and where the system still needs a human's help.
Have you tried automating parts of your issue-resolution workflow? I'd be curious to hear what's worked (or hasn't) for your team.
Ready to see how resolvo converts a GitHub issue into a tested PR in real time?
👉 Launch the Interactive Live Demo (No local setup required)