By Jinwu Xiao
This research presents a novel approach to manhole inspection that integrates high-speed 360° scanning technology with artificial intelligence to deliver comprehensive asset assessments. The methodology addresses critical challenges in underground infrastructure management by combining rapid data acquisition with AI-powered analysis for accurate, standardized condition reporting.
The system employs a specialized 360° camera that captures complete manhole interior data in approximately 15 seconds. Unlike traditional inspection methods requiring heavy equipment and stable mounting systems, this technology operates effectively despite camera movement and rotation, demonstrating robust performance under field conditions.
The captured video data undergoes 3D reconstruction to create comprehensive digital models of the manhole interior, which are then processed to generate “unfolded” images—flattened representations of the entire manhole wall surface that provide optimal viewing angles for defect detection and measurement.
The unfolded images serve as standardized inputs for trained large language models specifically developed for underground infrastructure assessment. These AI models identify and classify structural defects including cracks, corrosion, and deterioration, recognize and catalog infrastructure components and their conditions, provide calibrated measurements of defects and components, and generate reports in standardized formats compliant with industry standards.
Introduction: The Infrastructure Crisis and the Triangle of Inefficiency
Underground infrastructure constitutes the lifeline of modern civilization, yet it remains one of the most challenging assets to manage effectively. As sewer systems across North America and the globe continue to age, the need for frequent, accurate, and actionable inspection data has never been more critical. However, the industry is currently facing a bottleneck. The sheer volume of assets—specifically manholes—that require inspection vastly outpaces the capacity of traditional inspection methodologies.
Asset owners and engineers have long been forced to navigate a difficult compromise, which we classify as the “Triangle of Inefficiency.” This triangle is defined by three competing constraints: Time, Consistency, and Safety.
In the current paradigm, optimizing for one of these factors often degrades the others. For example, a thorough structural inspection (optimizing for Consistency) typically requires a crew to open a manhole, set up heavy tripods and lighting rigs, and deploy a CCTV camera or a laser scanner. This process is labor-intensive and keeps the crew exposed to traffic and hazardous gases for extended periods (compromising Time and Safety).
Conversely, a rapid visual inspection, where a worker simply looks down the hole with a flashlight or snaps a quick photo, saves time but results in highly subjective data that lacks the detail necessary for long-term rehabilitation planning (compromising Consistency). Furthermore, the human factor introduces significant variability. Two experienced inspectors looking at the same corroded bench or cracked chimney might code the defect differently based on their subjective interpretation or level of fatigue. This inconsistency makes it nearly impossible for municipalities to run accurate predictive models or allocate rehabilitation budgets effectively.
The “Missing Middle”: Redefining Inspection Levels
The underground utility industry generally categorizes manhole inspections into three distinct levels:
- Level 1: Visual Screening – This is a qualitative, rapid assessment. It is fast but superficial, often failing to capture structural nuances or measurements.
- Level 2: Documentation – This involves standard recording, usually via CCTV, to provide a video record. While better than a visual check, it still relies on manual review and does not typically offer structural dimensional data.
- Level 3: Structural Assessment – This is the gold standard, often utilizing LiDAR or advanced photogrammetry to build 3D models. It provides accurate, quantitative data but is slow, expensive, and computationally heavy.
There has long been a desire for a solution that sits in the “Missing Middle”—a technology that delivers the granular detail and accuracy of a Level 3 assessment but operates at the operational speed and cost-efficiency of a Level 1 screening. Our research at Purdue University, utilizing advanced computer vision and Large Language Models (LLMs), aims to fill this gap. By automating the data capture and interpretation phases, we can achieve Level 3 detail at Level 1 speed.
Methodology Step 1: Rapid Capture Technology
The foundation of our approach is the decoupling of data capture from data analysis. In traditional methods, the inspector analyzes the asset while they are at the site. In our workflow, the goal is to capture reality as quickly as possible and leave the analysis to the machine.
We utilize a lightweight, consumer-grade or prosumer-grade 360° camera (such as a GoPro 360 or similar Insta360 hardware). This choice of hardware is deliberate. Unlike specialized crawlers or heavy scanning rigs that require dedicated trucks and power supplies, a 360° camera is portable, battery-powered, and can be mounted on a simple pole.
The operational workflow is drastically simplified. A field technician opens the manhole, lowers the camera on a pole to the invert, and raises it back up. The camera captures the entire 360-degree environment in high definition video during this vertical transit. Because modern 360° cameras possess excellent internal stabilization, the system is robust against motion blur and rotation. The entire capture process takes approximately 15 seconds per manhole.
This speed has profound implications for field productivity. We have validated in field tests that a single crew can inspect approximately 100 manholes in a standard 4-to-6-hour shift. Furthermore, because the crew spends less than a minute at each open manhole, their exposure to traffic risks and potential falls is minimized, significantly enhancing safety.
Methodology Step 2: The Digital Transformation
Raw video footage from a 360° fisheye lens is difficult for humans to review efficiently. It requires the viewer to constantly pan, tilt, and zoom to see defects, meaning the review time often exceeds the recording time. Furthermore, raw video frames are inconsistent; the perspective changes constantly as the camera spins or tilts. This inconsistency makes raw video a poor input for Artificial Intelligence models, which thrive on standardized data.
To solve this, we employ a geometric transformation technique that “unfolds” the cylindrical interior of the manhole into a single, flat 2D image, often referred to as a “strip” or “unrolled” view. Imagine the manhole wall as the label on a soup can. Our software virtually slices this label and lays it flat. This process transforms the 3D reality into a standardized 2D plane. In this “unfolded” format, the vertical axis represents the depth of the manhole (from the rim to the invert), and the horizontal axis represents the 360-degree circumference.
This transformation is critical for two reasons. First, it allows a human engineer to view the entire condition of the manhole—from the chimney and corbel, down the walls, to the bench and channel—in a single glance without scrubbing through video. Second, and more importantly, it standardizes the data for the AI. Regardless of the manhole’s diameter or depth, the output is always a consistent, flat image where defects are presented in a predictable orientation.
Methodology Step 3: The AI Engine and Multi-Modal Analysis
Once the image is unfolded, the “Brain” of the system takes over. We utilize a sophisticated, multi-stage AI architecture that moves beyond simple object detection. While traditional computer vision might simply draw a box around a crack, our system aims to understand the context and severity of that crack, much like a human expert would.
The architecture relies on a combination of open-source segmentation models (SAM) and Google’s Gemini Large Language Models (LLM).
Segmentation (The “Where”) – First, the system scans the unfolded image to identify regions of interest. It segments out distinct components, such as steps, pipe inlets, the bench, and the channel. It also identifies anomalies that resemble defects, such as fractures, roots, or infiltration staining.
Interpretation (The “What”) – Identifying a blob on an image is not inspection. Inspection requires interpretation. This is where the Gemini Vision model is applied. We feed the AI the specific image segments along with contextual data. The AI analyzes the visual texture, color, and shape of the defect. For instance, it can distinguish between a harmless shadow and a severe fracture. It can differentiate between light surface rust on a step and deep corrosion that compromises structural integrity.
Clustering and Context – One of the challenges with AI is that it might identify a long crack as ten separate small cracks. Our system employs clustering algorithms (like DBSCAN) to group proximal detections. If the AI detects multiple fracture points in a vertical line, the system understands this is likely a single, continuous vertical fracture. This mimics the cognitive process of a human inspector who sees the “whole picture” rather than isolated pixels.
Methodology Step 4: The Rules Engine and Compliance
A common criticism of Generative AI is its potential to “hallucinate” or provide inconsistent results. In civil engineering, accuracy is non-negotiable. To address this, we do not rely solely on the AI’s output. Instead, the AI’s findings are passed through a strict validation filter—a “Rules Engine”—based on NASSCO’s MACP (Manhole Assessment Certification Program) standards ensuring high **Standards Compliance**.
This Rules Engine acts as a logic gate that enforces engineering constraints. For example, if the AI detects a defect and classifies it as a “Cover” defect, but the location data places it at the bottom of the manhole, the Rules Engine flags this as a logical impossibility and rejects or reclassifies the finding. Similarly, the engine ensures that the severity ratings (1 through 5) align with the visual evidence based on the strict definitions provided in the MACP codebook.
This hybrid approach—using the creative power of AI for detection and description, but constraining it with a rigid, logic-based Rules Engine—ensures that the final data is not only descriptive but also compliant with industry standards and ready for integration into asset management software.
Results: From Data to Actionable Insights
The output of this workflow is a comprehensive Asset Intelligence Report. This report is generated automatically, reducing the office processing time from hours to minutes. The system delivers on the promise of the “Missing Middle” through several key performance indicators:
- Speed and Efficiency: Field capture time is reduced to approximately 15 seconds per asset. Total processing time, from upload to report generation, is roughly 5 minutes per manhole. This represents a time reduction of approximately 70% compared to traditional Level 2 or Level 3 inspections.
- Scalability: The processing architecture is cloud-based and stateless. This means the system can scale horizontally. Whether a municipality needs to process 10 manholes or 10,000 manholes, the system can spin up the necessary computing resources to process them simultaneously.
- Auditability: Trust is paramount in AI adoption. Unlike “black box” solutions that just spit out a grade, our system provides full auditability. Every defect identified in the report is linked to a specific, high-resolution image segment. A human reviewer can click on a defect code in the report and instantly see the corresponding portion of the manhole wall. This “human-in-the-loop” capability allows for rapid verification and builds confidence in the automated results.
Future Outlook and Continuous Learning
The integration of 360° scanning and AI represents a paradigm shift in underground asset management. However, the technology is still evolving. During our testing and development, we have identified several areas for future enhancement.
One significant advantage of this AI architecture is its ability to learn continuously. As the system processes more manholes across different geographies and conditions—brick, pre-cast concrete, block—the underlying models become more refined. The segmentation masks become tighter, and the defect descriptions become more nuanced. We are effectively building a global knowledge base of infrastructure conditions that grows smarter with every inspection.
We are also expanding the application of this technology beyond vertical manholes. Early tests indicate that similar unfolding and AI analysis techniques can be applied to large-diameter pipes and culverts, providing a unified solution for sewer system assessment. Furthermore, we are addressing the challenges of absolute measurement. While the current system provides excellent relative positioning, integrating Lidar or photogrammetric depth data into the 360° video stream will allow for sub-millimeter measurement accuracy in the future, further cementing this technology as a true Level 3 equivalent.
Conclusion
The aging infrastructure crisis requires solutions that are exponential, not incremental. We cannot solve a 21st-century problem with 20th-century tools. By combining the accessibility and speed of consumer-grade 360° cameras with the analytical power of modern multi-modal AI, we are moving closer to a future where underground infrastructure assessment is not just faster and cheaper, but fundamentally more intelligent. This technology democratizes advanced inspection, allowing municipalities of all sizes to obtain the data they need to protect their communities and the environment.
Jinwu Xiao is a Ph.D. student and Research Assistant at Purdue University whose research focuses on AI-driven 3D scanning, digital twins, and intelligent infrastructure asset management, bridging academic innovation with next-generation industry applications.
