A compiled reference of 29 verified statistics on computer-vision damage and defect detection, drawn only from peer-reviewed papers, arXiv preprints, public benchmark datasets, and government research. No vendor marketing figures appear on this page. It covers what models actually score on obvious damage, where they still fail on subtle damage, how small the standard benchmarks really are, the measurement traps that inflate published accuracy, and the one number every accuracy claim should be read against: how much trained human inspectors disagree when they look at the same thing.
Key statistics
8 highlights from this report
Key statistics
Key takeaways
The published record is consistent across three unrelated domains: satellite imagery, vehicles, and factory surfaces. Computer vision is close to solved for damage that is large and high-contrast, and still weak for damage that is small, faint, or ambiguous. The honest comparison is not against perfection but against human visual inspection, which is measurably inconsistent.
49 credentialed human inspectors rating the same bridges produced up to five different condition ratings.
Only 68% of human condition ratings land within one point of the average; 95% within two.
On vehicle damage, the strongest baseline scored 92.6 mask AP on glass shatter and 9.8 on cracks.
The xBD baseline scored 0.66 F1 on undamaged buildings and 0.0094 on major damage.
A model that says "no damage" to everything still scores 75% accuracy on xBD.
On clean industrial defects, state of the art reaches 99.6% image-level AUROC.
On harder industrial scenarios, state-of-the-art defect localization stays below 60% AU-PRO.
A deep network detected concrete cracks down to 0.04 mm, versus 0.1 mm for classical methods.
How we built this report
Every figure was compiled in August 2026 from published research and public benchmarks, read at the original document, and checked before publishing.
- Published research only
Peer-reviewed journal and conference papers, arXiv preprints, official benchmark dataset documentation, and one US federal research report. We deliberately excluded vendor accuracy claims, including our own, because the point of this page is evidence a skeptic would accept.
- Read at the source
Each paper was downloaded and its text extracted locally, so every figure here was read in the paper or its table rather than taken from a summary. Preprints are labeled as preprints. Where a paper sits behind a paywall, we read the authors' arXiv copy and cite the journal of record.
- Failures reported alongside successes
Where a paper reports both a strong headline number and a weak per-class result, both appear. Several of the most useful figures on this page are the ones showing where these systems do not work.
- Independent review
Written by one co-founder, reviewed by the other before publishing.
These figures come from different domains (satellite imagery, concrete, steel, vehicles, factory parts) and different tasks (classification, detection, segmentation), so they are not directly comparable to each other and none of them is a prediction about any specific product's performance on your property. Benchmark scores are also measured on benchmark data; field conditions are harder.
The published evidence, by the numbers
All 29 figures, grouped by theme, each from a named source and independently verified.
The human baseline
Every accuracy claim in this field implies a comparison, and the comparison is almost never stated. The best public measurement of inter-rater agreement in structural condition assessment comes from the Federal Highway Administration (dot.gov), which had 49 credentialed inspectors from 25 state agencies perform ten inspection tasks on seven test bridges. Condition ratings for the same structure "varied over a range of up to five different ratings," and the study predicted only 68 percent of ratings would fall within one point of the average. It also found that in-depth visual inspections "are not likely to detect or identify the specific types of defects for which the inspection is prescribed."
Statistic 1
Condition ratings assigned by 49 credentialed bridge inspectors from 25 state agencies varied over a range of up to five different ratings for the same structure.
Federal Highway Administration, reliability of visual inspection study (2001)
Statistic 2
Only 68 percent of condition ratings were predicted to fall within one rating point of the average, and 95 percent within two points.
Federal Highway Administration (2001)
Statistic 3
In-depth inspections using visual inspection alone are not likely to detect or identify the specific types of defects for which the inspection is prescribed.
Federal Highway Administration (2001)
Statistic 4
Factors correlating with inspection results included fear of traffic, visual acuity and color vision, light intensity, and how rushed the inspector felt.
Federal Highway Administration (2001)
What this means: the relevant question is not whether automated detection is perfect, but whether it is more consistent than a person working under time pressure in poor light. Two caveats belong with this number. The study is about bridges, not guest rooms, and it is 25 years old. And these are 0-to-9 condition ratings for an entire bridge component, an aggregate judgment more subjective than the item-level calls a property inspection makes, so treat the disagreement figure as an upper bound rather than a direct analogue. We cite it because it is the most rigorous public measurement of inter-rater agreement in visual condition assessment we could find, and nothing comparable exists for property condition.
What models score on obvious damage
Where damage is large, high-contrast, and photographed close up, published results are strong. On the MVTec industrial anomaly benchmark, PatchCore reports (arxiv.org) an image-level detection AUROC of up to 99.6 percent. In a peer-reviewed concrete study, Dorafshan, Thomas and Maguire (und.edu) found a deep network "accurately labeled" crack images "with 99% accuracy," though on a test set only 9.3 percent cracked, where always answering "no crack" would score 90.7 percent. And in building CarDD (arxiv.org), the authors trained a filter to screen scraped photos for damage; on a balanced 1,000-image test set it reached 94.3 percent accuracy with 97.6 percent recall.
Statistic 5
PatchCore reports an image-level anomaly detection AUROC of up to 99.6% on the MVTec AD benchmark, more than halving the error of the next best method.
Roth et al., CVPR 2022 (arXiv preprint of the accepted paper)
Statistic 6
A deep convolutional network labeled concrete crack images with 99% accuracy on a test set that was only 9.3% cracked (319 of 3,420 sub-images), where always answering "no crack" scores 90.7%; in transfer learning mode it detected about 86% of the cracked images. Separately, classical edge detectors identified 53 to 79% of cracked pixels, a pixel-level measure not comparable to the image-level accuracy figure.
Dorafshan, Thomas and Maguire, Construction and Building Materials (2018)
Statistic 7
In transfer learning mode a deep network detected concrete cracks wider than 0.04 mm; the classical edge detectors were useful only above 0.1 mm.
Dorafshan, Thomas and Maguire, Construction and Building Materials (2018)
Statistic 8
A filter built to screen scraped photos for car damage achieved 94.3% accuracy, 91.6% precision and 97.6% recall on a balanced 1,000-image test set, the easiest form of the task and on a 50/50 class split.
Wang et al., CarDD, IEEE Transactions on Intelligent Transportation Systems (2022); data-cleaning stage, not evaluated on CarDD itself
Statistic 9
A two-stage model reached a combined F1 of 0.66 on the xView2 building damage task, against a challenge baseline of 0.28 (the combined localization-plus-damage metric, not directly comparable to the 0.2654 damage-classification baseline below).
DeepDamageNet, arXiv preprint (2024), not peer-reviewed
Statistic 10
A YOLO-v7 segmentation model reported 96.1% mAP at 0.5 IoU on concrete cracks and spalling while running at 40 frames per second, against 92.1% at 18 FPS for Mask R-CNN.
Ataei et al., arXiv preprint (2025), not peer-reviewed; 10,995 images augmented from 400 originals, so if the split was made after augmentation the test set contains transforms of training images. mAP at 0.5 IoU is also far more permissive than the COCO-style averaging used for CarDD
What this means: the 99 percent figures are real but narrow. Two of the three come from tasks that only ask "is anything wrong in this image," which is far easier than finding where and how bad. The YOLO figure is a preprint whose training set was augmented from 400 original photographs, so treat it as promising rather than settled.
Where it still fails: the subtle-damage gap
This is the most consistent finding in the literature, and it replicates across three unrelated domains. On vehicles, the CarDD authors report their strongest baseline detector reaching 92.6 mask AP on glass shatter and 9.8 on cracks, with their own improved model lifting cracks only to 16.6. On satellite imagery, the xBD baseline scored 0.6631 F1 on undamaged buildings and 0.4657 on destroyed ones, but 0.1435 on minor damage and 0.0094 on major damage. On factory surfaces, MVTec's own 2025 follow-up benchmark reports that state-of-the-art methods remain "below 60% average AU-PRO" once scenarios include transparent objects, difficult lighting, and extremely small defects.
The same pattern, three cuts of the data
Published per-class scores, obvious damage against subtle damage. All bars are drawn on a common 0-to-100 scale, but the underlying metrics differ: the CarDD rows are COCO-style mask average precision averaged over IoU 0.50 to 0.95, and the xBD rows are per-class F1 on a 0-to-1 scale. Bar lengths are comparable only within a block, never across blocks. The third block is a second view of the CarDD data, cut by object size rather than damage type.
Compiled by RapidEye Research from published tables: CarDD per-category and per-scale average precision (Wang et al., IEEE Transactions on Intelligent Transportation Systems, 2022) and the xBD baseline per-class F1 (Gupta et al., 2019). The xBD figures are the dataset paper's own baseline model, whose authors attribute the collapse to a weak imbalanced training regimen; it is not state of the art, and the shape of the gap, not the absolute level, is the finding.
Statistic 11
On CarDD, the strongest baseline detector (DCN, ResNet-101) scored 92.6 mask AP on glass shatter against 9.8 on cracks, a roughly nine-fold gap; the paper's own improved model raised cracks only to 16.6.
Wang et al., CarDD, IEEE Transactions on Intelligent Transportation Systems (2022), per-category results table
Statistic 12
Across five widely used detection models on CarDD, none exceeded 52.5 mask AP or 54.3 box AP (COCO-style, averaged over IoU 0.50 to 0.95); the paper's own improved model reached 57.0 and 60.6.
Wang et al., CarDD (2022), comparison table; models tested were Mask R-CNN, Cascade Mask R-CNN, GCNet, HTC and DCN
Statistic 13
Small objects account for over 90% of crack instances, over 45% of scratches, and over 35% of dents in CarDD, and the strongest baseline scored 19.7 mask AP on small objects against 66.3 on large ones.
Wang et al., CarDD (2022)
Statistic 14
The xBD baseline scored 0.6631 F1 on undamaged buildings and 0.4657 on destroyed, but 0.1435 on minor damage and 0.0094 on major damage; on major damage its precision was 0.7259 against recall of 0.0047, meaning it was usually right when it fired and almost never fired.
Gupta et al., xBD dataset paper (2019), baseline results table
Statistic 15
On MVTec AD 2, eight harder industrial scenarios totaling more than 8,000 images, state-of-the-art anomaly detection methods remain below 60% average AU-PRO.
Heckler-Kram et al., MVTec Software, arXiv preprint (2025)
Statistic 16
On a controlled road-crack benchmark, the best crack-specific segmentation model reached only 70.6 F1 and 54.6 IoU at the pixel level.
UDTIRI-Crack benchmark, arXiv preprint (2025), not peer-reviewed
What this means: the failure mode is consistent and physical. Thin, low-contrast, small-in-frame damage is hard, and grading severity is harder than spotting presence. Severity grading is measurably harder than presence detection: every benchmark here scores worse at "how bad" than at "is anything wrong."
The measurement traps
Two published caveats explain most of the variance between confident vendor numbers and disappointing field results. The first is class imbalance: the xBD authors note plainly that "a classifier that predicted 'no damage' on all of the images would retain 75% accuracy," which is why the benchmark reports weighted F1 instead. The second is resolution: the CarDD authors report their images average 684,231 pixels, 13.6 times the resolution of the prior public car-damage dataset, and identify small object size as the core reason cracks and scratches are missed.
Statistic 17
On the xBD damage dataset, a model predicting "no damage" for every image would still score 75% accuracy, which is why the benchmark reports weighted F1 instead.
Gupta et al., xBD dataset paper (2019)
Statistic 18
CarDD images average 684,231 pixels, 13.6 times the resolution of the prior public car damage dataset at 50,334 pixels.
Wang et al., CarDD (2022)
Statistic 19
The baseline model in the xBD dataset paper scored an overall weighted F1 of 0.2654 on damage classification.
Gupta et al., xBD dataset paper (2019)
Statistic 20
Performance on the original MVTec AD and VisA benchmarks has saturated, with state-of-the-art models often separated by less than one percentage point of segmentation AU-PRO.
Heckler-Kram et al., MVTec Software, arXiv preprint (2025)
What this means: when a vendor quotes a single accuracy percentage without naming the task, the class balance, and the image resolution, that number is close to uninterpretable. The useful questions are which damage types were tested, at what capture quality, and what the false-positive rate was on undamaged items.
How big the benchmarks actually are
The datasets behind these results are smaller than the field's confidence implies. The largest by annotation volume is xBD (arxiv.org), with 850,736 labeled building polygons across 22,068 satellite images and 45,361.79 square kilometers; SDNET2018 (nih.gov) has the most images at over 56,000, though each is a small labeled patch. After that the numbers fall quickly: DAGM has 16,100 synthetic images, and several of the most-cited crack benchmarks are in the hundreds.
Statistic 21
The xBD dataset contains 22,068 satellite images from 19 natural disasters with 850,736 labeled building polygons covering 45,361.79 square kilometers.
Gupta et al., xBD dataset paper (2019)
Statistic 22
SDNET2018 contains over 56,000 images of cracked and non-cracked concrete bridge decks, walls and pavements.
Dorafshan, Thomas and Maguire, Data in Brief (2018), peer-reviewed and open access
Statistic 23
SDNET2018 includes cracks as narrow as 0.06 mm and as wide as 25 mm, plus deliberate obstructions including shadows, surface roughness, scaling, edges, holes and background debris.
Dorafshan, Thomas and Maguire, Data in Brief (2018)
Statistic 24
The DAGM industrial optical inspection benchmark contains 16,100 images across 10 defect classes, but is artificially generated and each image contains at most one defect.
Carvalho et al., benchmark review, arXiv preprint (2023)
Statistic 25
The NEU steel surface defect database contains 1,800 grayscale images at 200 by 200 pixels across six defect types, contains no defect-free items, and has no default train and test split.
Carvalho et al. (2023), reviewing the 2013 Northeastern University dataset
Statistic 26
CarDD, the first large public vehicle damage dataset, contains 4,000 high-resolution images with over 9,000 annotated instances across six damage categories.
Wang et al., CarDD, IEEE Transactions on Intelligent Transportation Systems (2022)
Statistic 27
The CrackForest benchmark contains 118 road crack images, and DeepCrack contains 537 concrete and asphalt images at 544 by 384 pixels.
UDTIRI-Crack benchmark, arXiv preprint (2025), describing both datasets
Statistic 28
MVTec AD 2 comprises eight anomaly detection scenarios with more than 8,000 high-resolution images, including transparent and overlapping objects, dark-field and backlight illumination, and extremely small defects.
Heckler-Kram et al., MVTec Software, arXiv preprint (2025)
Statistic 29
Of SDNET2018's 56,092 images, only about 8,484 contain cracks, an imbalance typical of real defect data.
Dorafshan, Thomas and Maguire, Data in Brief (2018); our arithmetic from the published per-category counts
What this means: published accuracy is measured on datasets ranging from a hundred images to tens of thousands, often collected under favorable conditions. That is enough to establish that the approach works and where its edges are; it is not enough to predict how any system behaves on a specific property's photos, which is why a pilot on your own images is worth more than any benchmark number.
How to read these metrics
Six measures appear on this page and they are not interchangeable. This is the short version, because a percentage means nothing here without knowing which one it is.
- Accuracy
The share of images classified correctly. Badly misleading when most images are undamaged: on xBD, answering "no damage" every time scores 75%.
- Precision and recall
Precision is how often a flag is real; recall is how much of the real damage got flagged. A system can have high precision and near-zero recall, which is exactly what the xBD baseline did on major damage.
- F1
The harmonic mean of precision and recall, on a 0-to-1 scale. Used by xBD because it survives class imbalance.
- AUROC
How well a model ranks damaged above undamaged across every possible threshold. Threshold-free, so 99.6% AUROC does not mean 99.6% of defects are caught at any particular operating point.
- Average precision (AP) and mAP
Detection and segmentation quality, accounting for both what was found and where. COCO-style AP averages over IoU thresholds 0.50 to 0.95; mAP at 0.5 IoU only requires loose overlap and produces much higher numbers, so the two should never be compared directly.
- AU-PRO
A localization measure used in industrial anomaly detection, rewarding correct defect regions rather than correct image labels.
Cite this study
Academic or press use: copy a ready-made reference. RapidEye is the publisher.
Quick FAQ
How accurate is AI at detecting damage from photos?
It depends entirely on how obvious the damage is, and the published range is enormous. On clear, high-contrast defects, results are excellent: PatchCore reports up to 99.6% image-level AUROC on the MVTec anomaly detection benchmark, a deep network labeled concrete crack images with 99% accuracy in a peer-reviewed comparison, and a filter built to screen photos for car damage reached 94.3% accuracy on a balanced test set. On subtle damage the same techniques collapse: on the CarDD vehicle dataset, the strongest baseline detector scored 92.6 mask AP on glass shatter and 9.8 on cracks, and the paper's own improved model raised cracks only to 16.6; on the xBD building damage benchmark the published baseline scored 0.0094 F1 on major damage.
Why do AI damage detection accuracy claims vary so much?
Three reasons. First, class imbalance: the xBD dataset paper notes that a model predicting no damage on every image would still score 75% accuracy, so headline accuracy is inflated whenever most of what you photograph is undamaged. Second, task difficulty: deciding whether an image contains damage is far easier than locating it, and far easier again than grading its severity. Third, object size: in the CarDD dataset small objects account for over 90% of crack instances, and the strongest baseline scored 19.7 mask AP on small objects against 66.3 on large ones.
How reliable are human inspectors, for comparison?
Less reliable than most people assume. In the Federal Highway Administration's study of visual bridge inspection, 49 credentialed inspectors from 25 state agencies rated the same bridges and condition ratings varied over a range of up to five different ratings; the report predicted only 68% of ratings would fall within one point of the average and 95% within two. It also found that in-depth visual inspections are not likely to detect the specific defects they are prescribed to find, and that outcomes correlated with factors including fear of traffic, visual acuity and color vision, light intensity, and how rushed the inspector felt.
How large are the damage detection benchmark datasets?
Smaller than the field's confidence suggests. The largest by annotation volume is xBD, with 850,736 labeled building polygons across 22,068 satellite images; SDNET2018 has the most images at over 56,000 small concrete patches, DAGM 16,100 synthetic images, and MVTec AD 2 more than 8,000. Several widely cited benchmarks are tiny: the NEU steel surface dataset is 1,800 images, DeepCrack is 537, and the CrackForest dataset is 118 images. CarDD, the first large public vehicle damage dataset, is 4,000 images with over 9,000 annotated instances.
What is the smallest damage computer vision can detect?
In a peer-reviewed concrete study, a deep network in transfer learning mode detected cracks wider than 0.04 mm, against 0.1 mm for the classical edge detectors. That is a laboratory floor on high-resolution close-range imagery, not a field guarantee: the same body of research shows detection degrades sharply with distance, low contrast, and small object size, and resolution is a binding constraint. The CarDD authors note their images average 684,231 pixels, 13.6 times the resolution of the prior public car damage dataset.
Data sources
Every figure on this page traces to one of these named sources. Where the journal version is paywalled, we read the authors' arXiv copy and name the journal of record. Preprints are labeled as preprints in the statistic cards; no vendor performance claims are included.
Federal Highway AdministrationResearch report on the reliability of visual inspection for highway bridges, 2001dot.gov
arXivPreprints and accepted-paper copies: xBD, CarDD, PatchCore, MVTec AD 2, DeepDamageNet, UDTIRI-Crack, defect-benchmark review, concrete segmentationarxiv.org
PubMed CentralOpen-access copy of the SDNET2018 concrete crack dataset paper, Data in Brief, 2018nih.gov
University of North DakotaInstitutional repository record for the concrete crack detection comparison, Construction and Building Materials, 2018und.edu
MVTec SoftwareIndustrial anomaly detection benchmark documentation and its 2025 successor benchmarkmvtec.com
IEEETransactions on Intelligent Transportation Systems, publisher of the CarDD vehicle damage paper, 2022ieee.org