⚖️
PatchCourt
Verified Auto-Fix Pipeline
run 2025-01-15T14-32-00

Patches only ship when an independent verifier proves they work.

PatchCourt uses isolated Bob subagents to separate "wrote the fix" from "judged the fix" — the architectural gap that makes AI auto-fix actually trustworthy. Built for the IBM Bob 2.0 Hackathon.

IBM Bob 2.0 4 Subagent Roles fork_context: false 10/10 Tests Passing
3/3
VERDICTS ACQUITTED
0 rejected this run
17
Total Findings
5 security · 12 coverage
3
Patches Proposed
3 selected from 17
3
Acquitted
avg confidence 0.98
0
Rejected
no bad patches this run
5m
Wall Clock
vs ~45 min manual
⟳
Pipeline Architecture
Orchestrator
Bob Agent mode
→
Auditors × 2
parallel
general · read-only
→
Fixer × 3
general · file-scoped
→
Verifier × 3
fork_context: false
explore · isolated
→
PR ✓
acquitted only
🔒
Context isolation enforced at the API level — not by prompt convention. Each Verifier was spawned with subagent_type: explore and fork_context: false. The orchestrator projected {finding, diff, test_before, test_after} only — the rationale field is architecturally absent from every Verifier context window. The Verifier cannot write files. It judges only the diff and the test output it is given. This is the constraint that makes the trust claim real rather than theatrical.
⏱
5m 12s
Pipeline wall clock
🧪
7 → 10
Tests passing (before → after)
👤
~40 min
Manual review saved
🤖
8
Bob subagents spawned
🔍
Findings → Patches → Verdicts
F-SEC-001 Arbitrary code execution — eval() on caller-supplied input High Security ✓ ACQUITTED ▼
Location
src/hash_map.py : 31
Evidence
return eval(expression)
Patch diff — authored by Fixer subagent (rationale stripped before Verifier saw this)
--- a/src/hash_map.py
+++ b/src/hash_map.py
@@ -24,9 +24,6 @@
     def compute_key(self, expression):
-        # NOTE (security): evaluates a caller-supplied expression
-        # directly with eval(). Any input reaching this from outside
-        # the process (a config file, a request body, a CLI arg) is
-        # arbitrary code execution.
-        return eval(expression)
+        raise TypeError("compute_key does not support arbitrary expression evaluation; pass a literal key directly")
✅ VERIFIER VERDICT: PASS
Confidence
0.98
eval(expression) eliminated entirely. Replaced with unconditional TypeError. No eval or exec calls remain in codebase. No existing test calls compute_key() — method is unused. For a security finding, removing the vulnerable code path entirely is the correct tradeoff over wrapping it. No regressions.
before: 7/10 passed → after: 10/10 passed
F-SEC-002 Missing bounds check — negative index aliasing & wrong exception type in find() Medium Security ✓ ACQUITTED ▼
Location
src/union_find.py : 13
Evidence
if self.parent[x] != x: # no bounds check — x=-1 silently aliases parent[-1]
Patch diff
--- a/src/union_find.py
+++ b/src/union_find.py
@@ -8,9 +8,8 @@
     def find(self, x):
-        # NOTE: no bounds validation on x. Negative indices silently
-        # alias to the wrong element (Python negative indexing) instead
-        # of raising, and out-of-range positive indices raise a raw
-        # IndexError instead of a clear, catchable error.
-        if self.parent[x] != x:
+        if not (0 <= x < len(self.parent)):
+            raise ValueError(f"x={x} is out of range [0, {len(self.parent)})")
+        if self.parent[x] != x:
             self.parent[x] = self.find(self.parent[x])
         return self.parent[x]
✅ VERIFIER VERDICT: PASS
Confidence
0.98
Guard added as first statement: 0 <= x < len(self.parent) catches negative indices (0 <= x fails) and out-of-range positive indices (x < len fails). Both cases now raise ValueError as the tests require — not the previous raw IndexError. Both previously-failing tests now pass. Recursive path-compression call is protected — parent[] only ever stores valid indices. No regressions.
FAILED: test_find_rejects_negative_index FAILED: test_find_rejects_out_of_range_index → 10/10 passed
F-COV-001 Wrong eviction order — LRU cache evicts MRU node (head.next) instead of LRU node (tail.prev) High Coverage ✓ ACQUITTED ▼
Location
src/lru_cache.py : 51
Evidence
evict = self.head.next # MRU side — silently corrupts cache under load
Patch diff
--- a/src/lru_cache.py
+++ b/src/lru_cache.py
@@ -43,11 +43,7 @@
         node = _Node(key, value)
         self.cache[key] = node
         self._add_front(node)
         if len(self.cache) > self.capacity:
-            # NOTE: evicts the node next to head (the most-recently-used
-            # side) instead of the node next to tail (the
-            # least-recently-used side) -- silently evicts the wrong
-            # entry under sustained load.
-            evict = self.head.next
+            evict = self.tail.prev
             self._remove(evict)
             del self.cache[evict.key]
✅ VERIFIER VERDICT: PASS
Confidence
0.99
List structure: head (sentinel) ↔ MRU ↔ … ↔ LRU ↔ tail (sentinel). _add_front() inserts after head, so tail.prev is always the least-recently-used node. Eviction correctly changed from head.next (MRU) to tail.prev (LRU). test_eviction_removes_least_recently_used was FAILED — now PASSED. No regressions.
FAILED: test_eviction_removes_least_recently_used → 10/10 passed
✗
Rejection Log
📋
No patches rejected in this run
All three proposed patches were independently verified as correct by the Verifier subagent. The rejection log is a first-class output of PatchCourt — a Verifier catching a bad patch with concrete evidence is the expected demo moment, not a failure case.
# To generate a rejection for demo:
# 1. Revert src/lru_cache.py line 47: change tail.prev → head.next
# 2. Run: python -m pytest tests/test_lru_cache.py -v (1 FAILED = "before")
# 3. Spawn Verifier with that diff + failing test output
# 4. Verifier returns verdict: "fail" — patch looks right but evicts wrong node
# 5. Show side-by-side with the correct ACQUITTED verdict above
🧪
Test Suite — Final State
10 tests collected · 10 passed · 0 failed All Green
tests/test_hash_map.py::test_put_getPASSED
tests/test_hash_map.py::test_overwritePASSED
tests/test_hash_map.py::test_missing_key_raisesPASSED
tests/test_lru_cache.py::test_basic_get_putPASSED
tests/test_lru_cache.py::test_eviction_removes_least_recently_usedPASSED ← fixed by F-COV-001
tests/test_lru_cache.py::test_zero_capacity_rejectedPASSED
tests/test_union_find.py::test_basic_unionPASSED
tests/test_union_find.py::test_transitive_unionPASSED
tests/test_union_find.py::test_find_rejects_negative_indexPASSED ← fixed by F-SEC-002
tests/test_union_find.py::test_find_rejects_out_of_range_indexPASSED ← fixed by F-SEC-002
📄
Run Record
Run IDrun-2025-01-15T14-32-00
Branchpatchcourt-demo
Auditors2 subagents · parallel · fork_context: false
Fixers3 subagents · general type · file-scoped writes
Verifiers3 subagents · explore type · fork_context: false
Rationale stripped✓ before every Verifier spawn
Wall clock312 seconds (5m 12s)
Manual equivalent~45 min (15 min × 3 findings)
PR descriptionPR_DESCRIPTION.md