
The evaluation used the customer’s manually annotated drawing as a benchmark, a sheet carrying 39 highlighted marks across 31 change zones. The proof of concept focused on identifying and prioritizing likely revision areas to support engineering reviews, not on determining or validating the exact changes.
- Benchmark-Based Evaluation: Image comparison, the two change detectors and OCR text matching were each scored on how many marked zones they found and how many false alarms they raised.
- Two Detectors, Ranked Results: Used together, the detectors caught 26 of the 31 marked zones. Zones flagged by both were ranked higher, so the most likely changes appear first.
- Highlighted, Confidence-Ranked PDF: The latest revision was returned with candidate change areas marked and ranked by detector confidence, giving engineers a visual starting point.
- Supporting Excel Workbook: For every flagged zone, the workbook shows its rank, whether the two detectors agreed, how much content changed and what text appears only in the current revision, so each flag can be checked against data.
- Limits Documented Openly: A Read-Me tab explains how the benchmark was run, how each method performed, how many false alarms it produced and where the approach falls short, so reviewers can judge its limits for themselves.
