Above: A view of the Brighton Dam in Brookeville, Maryland, from one of the project’s UAVs.
Combining UAVs and 3D computer vision can make infrastructure inspections cost-effective.
Data for the dam inspection was collected with both fixed-wing and multirotor UAVs.
More than 70% of the 90,000 dams scattered across the U.S. are, according to the American Society of Civil Engineers (ASCE), half a century old. The age of dams, plus their proximity to human communities, make the assessment of their structural fitness a critical job for civil engineers.
However, evaluating the condition of dams can be a challenging and time-intensive activity owing to their large-scale structures and complex designs. To be able to scrutinize an entire dam and get through hard-to-reach corners, inspectors require specialized equipment and training. There are also additional safety risks to inspectors, as many dams are located over large bodies of water.
The combination of all these factors makes dam inspections expensive.
So how about coupling the latest photogrammetric and UAV technologies to trim down dam-inspection costs and improve the safety of inspectors?
The HPCG Technique
That is what Dr. Ali Khaloo from Cornell University and Dr. David Lattanzi from George Mason University had in mind when they developed the Hierarchical Point Cloud Generation (HPCG) technique, a 3D computer-vision method for processing images captured by multiple drones.
If you work in assessing critical infrastructure systems such as dams and bridges and you haven’t heard of the technology behind HPCG, you soon will. Their novel approach has won accolades from several technology competitions, and the HPCG technique was issued a U.S. patent last August.
With HPCG, civil engineers can now create high-quality 3D models of critical infrastructures at a fraction of the cost compared to conventional terrestrial laser scanning (TLS) approaches. The resulting HPCG 3D models are richer with details because drones can reach areas where scanners could not possibly be positioned.
According to Khaloo, who is an expert on autonomous sensing and intelligent condition assessment of structures, HPCG is “designed to integrate images captured by UAVs at a wide range of standoff distances from a large structure to form a complete 3D model.”
This approach works well for monitoring bridges and dams whose very locations pose logistical challenges for stationary scanners—but are feasible for camera-mounted UAVs because they can hover near the target features.
Team members of the project pose at the Brighton Dam site.
Mix of Experts
From the start of his academic career, Khaloo has collaborated with Dr. Lattanzi, associate professor of civil engineering and a specialist in remote sensing and data analytics. The two scientists previously worked together in imaging the Placer River Bridge in Alaska. This time they wanted to test the HPCG technique on a different infrastructure.
They teamed up with Dr. Adam Jachimowicz, systems engineer and researcher at George Washington University, to try their technology at the Brighton Dam in Maryland. Jachimowicz has much experience with imaging dams using UAVs, and he was interested in the project.
“Often buildings and infrastructure are assessed multiple times by different teams. Using UAVs can create a common data set that many parties can share,” says Jachimowicz.
Jachimowicz then sought the permission of the Washington Sewer and Sanitary Corporation, the owner of the Brighton Dam, that was also planning to rehabilitate the dam’s structure. The dam’s manager thought it would be useful to have a reliable 3D model of the entire infrastructure, while the scientists saw an opportunity to use their advanced computer vision analytics.
“The Brighton Dam is a common type of dam that was built in the 1940s and so is representative of many similar dams,” says Lattanzi. “The dam was accessible from a flight clearance point-of-view scale, and its scale was useful for stress-testing our methods,” he adds.
Together they worked with Charles Devaney, an expert UAV pilot from Arizona who has a graduate geography degree focusing on geospatial technology from the University of Hawaii at Manoa.
“I brought my skills as an experienced UAS pilot,” says Mr Devaney. “My background in photogrammetry and post image-processing helped determine what data we needed to collect to get the results we wanted.”
“Having a fantastic team of experts with diverse backgrounds was the key factor in the success of the project,” says Khaloo.
An orthomosaic aerial view of the Brighton Dam from the fixed-wing UAV.
Creating 3D Point Clouds
In order to conduct a thorough visual inspection of the Brighton Dam, the project team used a mix of multiple UAV platforms and computer vision technology to create high-resolution 3D point clouds of the dam and its surrounding environment.
According to the team, their project could be the first using multi-scale photogrammetric 3D reconstruction techniques in capturing the overall geometry of a large-scale complex gravity dam by deploying an array of UAVs for data acquisition.
“We used two different types of UAVs, fixed-wing and multi-rotor,” describes Devaney. One fixed-wing UAV was mounted with a 24.3-megapixel camera and was used to capture the global geometry of both the upstream and downstream faces of the dam.
“Two other multi-rotor UAVs were each mounted with a 12-megapixel camera to generate a series of oblique image networks from varying standoff distances.”
The UAV flights were then planned carefully in order to obtain images with greater than 80% overlap at specified standoff distances (from 2.5 to 10 m). This was done to minimize the ground sample distance and hence to maximize the spatial resolution of the pixels (1 mm) for the 3D reconstruction of the dam’s image. The three UAVs captured a total of 2,020 images.
The images acquired by the cameras mounted on the three UAVs were grouped into what is called an “image network” or photos that were collected from similar standoff distances. Images from each network were then integrated and merged into a global, multi-scale point cloud model using the HPCG algorithm that seamlessly matched the photos into one single reconstruction.
[For a more detailed technical description of how the HPCG process was used in this project, see their work published in the journal Frontiers in Built Environment (doi: 10.3389).]
With HPCG, Khaloo and his team were able to create highly detailed and dense 3D models of the Brighton Dam containing more than one-billion points. They then found out that the HPCG models have more cloud points than those generated with a laser scanner.
“Laser scanning is great, but it can be difficult to securely set the scanner in the necessary places, and they can be fairly slow,” says Lattanzi. “In our experience, photogrammetry can yield higher quality results.”
Khaloo shares that view. “Using multiple UAVs to capture image data at different standoff distances give better results. With TLS, the 3D model resolution is fundamentally limited by the precision of the acquisition instrument. On the other hand, photogrammetry does not have an inherent resolution limitation as it is driven by the ratio of pixel size to target object size.”
Aside from acquiring better image quality, Jachimowicz cites three important factors on the choice of UAVs and photogrammetry over laser scanners in dam inspections: speed, access and cost.
“It took us about 10% of the time to collect our data using UAVs, and we covered a larger portion of the structure,” he says. “And a dam is tricky because of the size, configuration, and presence of water, so it can be a challenge to find locations to set up the TLS. Using a UAV, you don’t have such issues. You can easily collect image data from almost any location you want.”
And for the cost, Jachimowicz adds that “a TLS costs around $100,000. Our UAS equipment is cheaper.”
According to Lattanzi, UAVs have two key advantages. “They are faster at collecting data than humans, and it’s easier for them to access hard-to-reach areas of a dam without expensive equipment. Both have economic benefits.”
A generated 3D model of the Brighton Dam from downstream.
Replication
Global projections on the use of UAVs in the construction industry have been valued at $11.2bn by Goldman Sachs. Yet only a small portion is spent ($1.1bn) on drones for infrastructure inspection. This should soon change with the upsurge of better and more affordable drones in the market.
“I see UAVs becoming an intricate part of dam inspections,” says Devaney, the team’s UAS expert. “The main advantage of UAVs is the safety component. Not having to put a human in a dangerous situation is important.”
Lattanzi, on the other hand, believes that piloting protocols will become standardized in the future. “UAV teams will be inevitable in dam inspections, as we want to minimize time on site,” he says.
He adds that pre-flight planning will always be important, as well as teamwork and communication. “The pilots, managing engineers, and data analysts all need to be on the same page.”
But can their approach be replicated at other dam sites?
“Absolutely,” says Jachimowicz, who has since created images of other dam sites. “The process can easily be generalized.” Khaloo agrees. “One of the main objectives of our project at Brighton Dam was to develop a systematic approach for UAV-based dam inspection.”
He is now the founder and CEO of Aren, a tech start-up born from the same computer vision technology behind the HPCG. Aren’s end-to-end AI-powered platform can reduce the costs of their customers’ asset maintenance up to 50% versus conventional methods.
Through the technology that Khaloo developed, it is now possible to detect, track, and predict damages and failures to provide a holistic view of the asset’s health and to de-risk the asset management sector.
Better payload, enhanced imaging capabilities, and lower costs. These are the improvements that have been coming out lately with drone technologies and computer vision. However, these elements should not be the main factors to consider when choosing to use UAVs and photogrammetry in dam inspections.
“The key challenge in infrastructure inspection is to develop analytical methods that are viable for a broad range of dams in a variety of environmental conditions,” asserts Lattanzi. “We must make sure that our data and analyses will lead to reliable assessments to ensure the safety of our maintenance engineers and the general public.”