Above: This screenshot is of CityMapper output.

A look at the hybrid airborne system that combines lidar with oblique and nadir imagery.

Over the last five years I’ve built and used a lot of 3D GIS and BIM models. Almost all of this has required the capture of stereo imagery and oblique imagery and a lot of photogrammetry work to hand-build the data in vector format. For a small city or town, this can take a team of 50 photogrammetrists a fair few months. Then you have to do a lot of verification as you sieve through the data, as it’s often inconsistent because everyone works in a different way.

What if there were a way to fly the data as you would normally, but then just press a button and it all magically converts into a 3D vector city? It sounds too good to be true, right? Leica Geosystems seems to have done it with their CityMapper system; xyHt investigates.

City Modeling Legacy Conundrums

Prior to these recent requests for proposals, I had been relying on a modeling company to build the data, and the process was quite arduous. I would have to specify an aerial survey that would need stereo imagery with specific camera requirements to ensure good overlap and quality, then check the imagery before providing it to the modeling company.

There was the option to use satellite stereo imagery, but this was often out of date or not at a high enough resolution, but definitely, it at least ticked all the camera requirements.

Once the modeling team had the stereo imagery, they would go through digitizing the data using photogrammetry software. It was a slow and cumbersome process, which, depending on flight windows, staff availability, and area of buildings required could take a few months—even for a relatively small area. Today with the speed at which cities change, the data could potentially be out of date even before it reaches the customer.

When these recent large proposals came in, we had to rethink how we captured and modeled the data. We looked at Dubai, and it would have taken a team of 100 modelers at least six months, if not longer. We looked at updating current models of London; did you know that all those cool 3D models of London are either not geospatially accurate or represent only a small portion of London?

We needed to find a solution that was more autonomous, would remove the inconsistencies of individual modelers, and be much faster. That was the conundrum I was facing in September last year.

Coincidentally, I was just about to travel over to Intergeo for xyHt last year when I heard about this new solution that Leica Geosystems had been working on. They called it CityMapper: a multi-camera and lidar hybrid sensor system that would provide aerial capture and conversion into 3D data models. It sounded too good to be true.

The Hybrid Sensor Solution

Unless you’re an “anorak” (British slang for someone deeply interested in niche subjects) like me, you wouldn’t be searching “3D modeling” among the U.S. patents, but if you did, you’d come across patent US15468057. Filed March 23, 2017, the abstract reads:

“A method and a hybrid 3D – imaging device for surveying of a cityscape for the creation of a 3D city model.

“According to the invention, lidar data is acquired simultaneously with the acquisition of imaging data for stereoscopic imaging, i.e., acquisition of imaging and lidar data in one go during the same measuring process.

“The lidar data is combined with the imaging data for generating a 3D point cloud for extraction of a 3D city model, wherein the lidar data is used for compensating and addressing particular problem areas of generic stereoscopic image processing, in particular areas with unfavorable lighting conditions and areas where the accuracy and efficiency of stereoscopic point matching and point extraction is strongly reduced.”

The patent outlines a single piece of equipment that can be used for aerial surveying of cities that captures lidar, stereo (nadir) imagery, and oblique imagery. Although the capture of the multiple formats isn’t new, the combination of it all into a single system that can be mounted on an airplane certainly is unique.

It seems deceptively compact: not much bigger than a traveling golf case (albeit much heavier), designed to mount easily to the type and size of the typical openings made in small (or large) aircraft to accommodate legacy camera systems.

The Basics

As it stands, the current system is comprised of a Leica RCD30 CH82 multispectral camera in NADIR, 80MP CCD size and 10,320 x 7,752 pixels; the maximum frame rate is 1.5 sec. There are four Leica RCD30 CH81m, 80MP oblique cameras with a viewing angle of 45°. The motion compensation is mechanical with in-flight direction. There’s a Leica Hyperion lidar unit with a laser wavelength of 1,064 nm and pulse repetition frequency of up to 700 kHz and scan speed programmable up to 100 Hz, with a field of view of 40°. The vertical accuracy is < 5 cm and the horizontal accuracy < 13 cm.

I’ll let the expert photogrammetrists, lidar surveyors, and aerial surveyors among you comment on how good or bad the sensors are, but please keep in mind that a compromise was needed to make it fit it into a single unit.

What makes it work well, though, is that it is all aligned and works in conjunction with one another to ensure the data has value, you see, as Hexagon provides companion software that turns individual sensors into a game changer—much greater than the sum of its parts. This software is HxMap.

Ingesting and Processing

Starting from data download, raw QC to basic data management, HxMap guides you through the sensor-specific processing steps, based on the input data. What’s really clever is that Leica Geosystems has built-in artificial intelligence (AI) and 3D modeling workflows so that, once the data has been processed and tidied, it can simply be converted to smart-city-ready formats at the touch of a button. Yes, you heard that right: once the data has been processed, you can simply hit a button and convert the data to a “reality capture” mesh or a vector LOD 2.2 CityGML format (or other formats such as Esri shapefiles).

To be clear, from one flight, you get your oblique, stereo and point cloud, and are able to extract your DSM, DTM, reality capture mesh, and 3D vector buildings.

Real-world Capture

U.S. Cities

Leica Geosystems partnered with Fugro (an international asset integrity and geo-intelligence solutions provider) to begin acquiring and processing data over multiple cities and coastal regions across the U.S. Using CityMapper, Fugro conducted aerial surveys over some of North America’s most densely populated urban centers in support of their geospatial mapping services.

The results of the airborne surveys were post-processed using the latest version of HxMap, the unified high-performance multi-sensor workflow. Within a single and familiar interface common to processing other Leica Geosystems airborne sensors, all typical data products, from orthophotos and oblique images to point clouds, 3D meshes, and models, can be quickly and efficiently produced.

The latest update of the common-sensor post-processing software adds lidar point cloud generation, calibration, strip matching, and viewing to the current image processing capability. HxMap version 2.2 also incorporates enhanced noise filtering for both CityMapper and Leica SPL100 (single-photon lidar) sensors.

Fugro program manager Mike Wernau told us, “HxMap v2.2 is extremely powerful, as it […] establishes a single environment to process our CityMapper oblique and nadir imagery, point cloud, and terrain data all at once.

“The products derived from the Leica RealCity [an airborne 2D and 3D reality capture solution] workflow serve as the foundation for our land and property solutions, delivering real value to our customers in state and local government, insurance, and land-use management.”

The City of Bordeaux

Among France’s most exciting, vibrant, and dynamic cities, Bordeaux is a port city on the Garonne River in southwestern France. Bordeaux is the world’s major wine industry capital; the wine economy in the metro area takes in 14.5 billion euros (almost $17 billion) each year.

In 2007, after nearly a decade of infrastructure upgrades, Bordeaux was recognized by UNESCO as a World Heritage site for its vast Enlightenment-era urban core. After Paris, Bordeaux has the highest number of preserved historical buildings of any city in France.

In 2017, APEI (Aéro Photo Europe Investigation), a French aerial survey company specializing in the airborne acquisition of geospatial data, worked with the Direction de l’Information Géographique (D.I.G.) de Bordeaux Metropole for Bordeaux’s city mapping project. APEI delivered aero-triangulated oblique images captured with Leica City- Mapper, and Bordeaux will use them to create a 3D city model. D.I.G. de Bordeaux Metropole is responsible for managing the mapping and GIS reference layers used in the Bordeaux metropolis, and the project was flown in only two days (March 10 and 11, 2017).

Bruno Callabat, CEO and technical manager of APEI, said, “The system uses Leica PAV100 and Leica FlightPro, which are part of the Leica Geosystems airborne common platform. It makes it really easy for our staff to operate, and the training has been performed in only two hours.

“We used the Leica PodLifter, which is specifically designed to lower the camera into the PAV100 to get full view for oblique capture. It was a very useful feature.”

Lidar Point Clouds

When you look at specific use cases and compare the results of an image-generated point cloud with a point cloud generated from fused imaging and lidar data, the benefits are clear. In many countries, up to 35% of urban areas are covered with trees. As such, the image-derived point clouds are very dense on the buildings; however, the digital elevation models of streets and surfaces under the trees are disturbed.

In contrast, the CityMapper’s lidar penetrates through the street-side trees and generates an accurate ground model with a flattened ground surface as well as flatter building walls, right down to sidewalk level. In areas with shadows on the buildings or with irregular surfaces, the building edges in the image-derived point cloud can be fluffy. With the fused imaging and lidar workflow, these edges are much sharper and the resulting building models are more accurate.

Sophistication in Simplicity

It’s a deceptively simple concept: fly once, get the best of both worlds—imaging and lidar. The strengths of each make up for the weaknesses of the other. While this may just be the first of such hybrid systems, it’s a complete solution, from the data capture hardware through the processing software.

The rush to develop 3D cities, keep them current, and create digital twins is driven by a harsh reality. It has become clear that using legacy methods and 2D planimetric-based planning, design, construction, and operations will not be able to keep up in providing more, smarter, greener, and cost-effective infrastructure required from the explosive growth in our world. It is heartening to see solutions that can at least speed up the processes to build the 3D digital foundations we need to move forward.

---

Digital Twins

In the quest to improve the efficiency of planning, designing, building, and managing everything—from a single building, a transportation system, or a utility to an entire city—the idea of creating a completely modeled twin of a physical system is catching on. The idealized digital twin is envisioned as a living, breathing, digital duplicate of the physical. With a twin, so many things are possible: immersive VR and AR interactions, navigation, and inspections. More importantly, the way we design, build, and operate could dramatically change.

To reach these lofty goals, the twin could be more than just a 3D model of the geometry of a building or city; it could be fed by live sensor data (utility SCADA, transactions, depreciation of elements, costs, scheduling, and more). For instance, people designing in such twins will have access to the contextual geometry around a subject site, current and precise.

Transactions during construction, feedback from as-built surveys, UAS data, and inspection reports would keep the twins up to date. Then, in operations and asset management, various contingencies and scenarios could be tested in a fully functioning twin.

We see the beginnings of twinning in BIM models (e.g., IFC standard), Leica RealCity models, Autodesk Revit models, Bentley iModels, Esri ArcGIS Urban and ArcGIS Indoors, and more. But the foundation for a twin at a city level must include imagery, both nadir and oblique, together with lidar to create 3D geometry. Depending on the application, outputs might include 3D clouds, meshes, and/or wireframes. CityMapper gathers this foundation data from a single hybrid sensor.

Series Navigation

<< Tracking Wildlife from the Sky