Above: A full 3D textured model of San Francisco produced through Skyline’s PhotoMesh 3D Reconstruction software and streamed for visualization and analytics in Skyline’s TerraExplorer.

Surveyors and photogrammetrists capture reality data. But to model, manage, and provide access to massive and complex datasets—to provide your clients with true 3D mesh reality capture—you’re going to need help.

Geospatial and land surveying data capture is steadily moving into the next phase: reality capture and modeling. At the heart of the reality model is the reality mesh, a photo-realistic 3D representation of the physical environment. The mesh is flexible and intuitive, allowing the information to scale from site-specific to entire cities and regions.

The reality-capture phase of creating a mesh utilizes tools familiar in concept, if not practice, to most geospatial practitioners. Global navigation satellite systems (GNSS) provide geodetic positioning. Imagery provides measurement and context. These images may be acquired from airborne sensors—manned or unmanned— and by terrestrial techniques. While not necessary for mesh creation, lidar in a multitude of formats greatly enhances the precision and accuracy of the reality mesh. Proper application of these tools can create a high-accuracy 3D model of existing conditions.

We live in a dynamic world, yet the built environment changes slowly. As data can be valid for decades, the use-once and file-away mentality is evolving into the capture-once and use-often paradigm. But this new modality is not without its challenges.

How do you store massive amounts of data—often in the terabytes (and some industry leaders are wrestling with petabytes)?

How are the stored data accessed and consumed? Will it be used in-house? Will it be shared with clients? Will it be hosted on local servers or made available in the cloud? What kind of security models and protocols will be used to control who may view or edit the data? If you’re sharing with clients, do they need to download the data (creating their own storage nightmare) and use potentially expensive and specialized tools to perform analyses?

Who owns the data? Is it the source provider? Is it the client? Does the online host have a right to use the data as they wish? Could it be some combination of any or all of those prospects?

How are the data gathered? Are they all sole-sourced? Can other data sources be incorporated? What mechanisms and safeguards are included in the process to ensure data reliability?

If you own the means of gathering reality data—GNSS, imagery, and lidar—do you have the tools to create and consume mesh data? Do your software solutions effectively combine mixed-source data into an accurate and cohesive mesh?

These are important questions. Reality modeling is no longer a niche, but a growing mainstream trend in the geospatial and land surveying professions. Your future relevance will hinge on your ability to understand these concepts, consider these questions, and make the right decisions for your company and your clients.

It’s Your World

In a modern office high-rise, with a view of the airplanes coming and going at the Dulles International Airport in Herndon, Virginia, is a company working to help you answer these questions. Skyline Software Systems, Inc. offers a suite of software tools, ranging from mesh creation to placing the whole world in your hands, or at least on the server of your choice. Skyline has actually trademarked (as relates to geospatial matters) the term “It’s Your World” as a company tag line.

Founded in 1997, Skyline was started as a gaming company. It soon became apparent that the underlying rendering engine was well suited for providing large-scale, high-resolution, textured 3D mesh models. Refocusing on the geospatial sector, Skyline has grown to be an international provider of 3D spatial data solutions. The United States makes 60% of Skyline’s client-base, with 50% of the U.S. market being government agencies.

Taking the helm of the company in 2016, Skyline’s president and CEO, Robert Peters, gives the company a clear purpose. “We use photo and lidar fusion to create holistic 3D models with intelligence.”

To meet those goals, Skyline has developed three primary tools in the Skyline ecosystem: PhotoMesh, TerraExplorer, and the Skyline Globe Server. Skyline is poised to introduce a new feature for their customers, the forthcoming Skyline Atlas, in May 2019.

PhotoMesh

For content creators, it might be difficult to embrace the mesh if you don’t have the tools to create the mesh. PhotoMesh is Skyline’s mesh-creation application. It allows imagery and optional lidar data to be combined to create high-resolution, photo-realistic 3D mesh models.

Modern computing has enabled the concept of turning standard 2D photographs into 3D models. The simplified explanation for this technique is the stitching of overlapping photos from different perspectives that allows the software to create 3D measurements of the elements captured within the photos by comparing related pixels and computing a stereo relationship. This process is also referred to as aerial triangulation, borrowing from its airborne photogrammetric origins, but the same process may be applied to terrestrial imagery. The application of these principals is quite complex.

For the best results, the number of unknowns needs to be limited. GNSS-enabled cameras assist in providing a general, if not precise, location of the photos. While this is not necessary to get results, location does help in the meshing process.

Overlapping photos are essential. The camera type, camera orientation, and target accuracy may all affect the amount of overlap, but a 70% or greater overlap between photos is common.

For optimal results, a combination of airborne nadir, airborne oblique, and terrestrial photography provide more data and additional context to potentially fill areas that might be obscured with a single approach. The airborne may be from a manned aircraft for larger regions or a UAS for smaller areas. Terrestrial images are best from standard DSLR cameras, but even a smartphone could be used to get results.

Few measurement techniques can compete with the precision and completeness of data from lidar. Most modern terrestrial scanners are rated at 2mm or better precision. PhotoMesh will incorporate lidar to enhance the 3D results provided by the images.

Ground control points provide the geodetic accuracy to the resulting model and serve as a confirmation for the model scale. The targets may be intentionally delineated and set prior to data collection (preferable) or may be photo-identifiable points that receive observation and valuing after the data collection.

The most demanding task in creating the 3D mesh is combining multiple data sources into a seamless 3D mesh model. PhotoMesh refers to this process as fusion and will allow the distribution of this resource-hungry process across multiple computers or processors, referred to as fusers.

Each fuser shares the processing load to accelerate the processing time, therefore reducing the time each computer is under a heavy processing load. A single site of less than 50 acres can be processed in a reasonable time with a single fuser. Several hundred acres will benefit from several fusers, while entire cities may require 10 or more fusers.

Full 3D mesh models can be produced from multiple sources of varying resolutions. Algorithms perform color balancing to enhance the model consistency and realism. Proprietary edge-detection routines further augment the model by providing crisp geometry along edges and at corners, minimizing the melted or smudged appearance that plagues many mesh models.

The results conform with Skyline’s open data approach. Models can be exported in industry-standard formats such as 3DML and OBJ. The exports may also include true rectified orthophotos, digital surface models (DSM), and RGB (colored) point clouds in the popular LAS format.

TerraExplorer

TerraExplorer is a 3D integrated environment allowing the seamless combination of multiple data sources. These data sources can include the PhotoMesh models, meshes and models from other sources, CAD data, geodatabases, web-mapping services, and other sources of geospatially enabled data. If this sounds like a geographic information system (GIS), it is: it’s a GIS with a central focus on 3D visualizations.

TerraExplorer comes with a variety of data-analysis tools, including measurement tools, profilers, line-of-site analysis, and viewshed queries. TerraExplorer also contains data simulations such as flood studies, solar studies, and plume analysis. All these analyses and simulations are performed in the 3D environment, providing a dynamic and visually compelling view of the results. Realism may be further enhanced with moving water, fountains, smoke, moving vehicles (both ground and airborne), and simulated weather.

For users comfortable with coding, TerraExplorer’s API is the gateway; the entire API reference guide is available on Skyline’s website. Programmers can create custom tools and analyses using many common coding languages including JavaScript, C#, and C++. Anything created by the Skyline customer remains the sole ownership of the customer, but if they wish to share their new tools with Skyline they may be considered for inclusion in future versions of TerraExplorer.

TerraExplorer is available in several versions and formats, including TerraExplorer Mobile and TerraExplorer Web. The web interface is particularly well suited to providing clients access to the TerraExplorer data and results from a standard browser without needing to install and maintain their own version of the Skyline software.

Skyline Globe Server

Bringing the Skyline environment together is the Skyline Globe Server (SGS). Unlike many of their competitors, Skyline provides the end user a license to use the SGS as they see fit. This is not a globe hosted as a SaaS (software as a service), but instead it is a global framework for hosting on the server and architecture of the licensee’s choosing.

Ultimately, the SGS is a portable and private cloud storage solution. It works effectively on a private file server, a cluster of servers, or popular cloud-hosted servers, such as Microsoft Azure or Amazon Web Services. This provides an unprecedented level of flexibility.

SGS users may host a single globe or could spawn and maintain multiple globes with different purposes and levels of security and access. Since the globe is privately licensed, it is possible to transfer ownership of the SGS. For instance, some customers have set up and populated a globe with regional data and then transferred the SGS and data to a municipality for continued use and maintenance by the municipality.

The SGS supports data streaming in many popular formats including WMS, WFS, 3DML, and Cesium. These streams are Open Geospatial Consortium-compliant, allowing access and use of the SGS with many popular GIS and CAD platforms.

SGS works best in conjunction with TerraExplorer and PhotoMesh. All three programs are stand-alone solutions, but together they are designed to maximize their integration in the Skyline format.

Many government and municipal contracts have clear guidelines on data ownership. It is important to understand the possible implications of using a third-party hosting environment. With the licensee being in full control of the SGS, all hosted data remains fully within the ownership of the host.

Permissions-based architecture adds another layer of security, allowing the host to control who may access the data. The user-access control systems provide the SGS administrator with the options to manage users, groups, and administrative roles. This granular approach provides the fine-tuning to control the server-side storage options and the client-side read and write permissions.

Skyline Atlas

Skyline announced in October 2018 that it was forming a strategic business alliance with Geomni, a Verisk business. The initial offerings by both companies give geospatial consumers reasons to be excited.

During a recent interview, Eatay Ben Shechter of Skyline put it best: “Skyline is providing the tools and 3D streaming service, and Geomni is bringing their vast and continuously updated database of imagery and lidar to the deal.”

Simply stated, Geomni will continue its business model of acquiring and updating aerial intelligence throughout the world—consisting of oblique images, nadir images, and airborne lidar. Geomni customers will have a new option to purchase 3D-rendered data through Geomni, as supplied by Skyline upon request. This data will be provided as a per-site, static download for hosting by the customer.

Skyline customers will also be offered a new service, the Skyline Atlas, that will provide a dynamic stream of 3D-rendered data, developed from the Geomni database and hosted by Skyline on their own SGS framework. This subscription service gives the Atlas customer access to all the 3D-rendered data as they’re made available and as they’re updated.

With datasets in the terabytes for a single city, the benefits of streaming the data and allowing Skyline to host it are clear. Customers will be able to access a rich 3D environment at 1:1,000 (1” = 100’) accuracy for city-mapping visualization or analysis.

Geomni will retain ownership of the source data. However, anyone subscribing to the Atlas will also be able to use the Geomni-rendered data to meet their project needs.

Derivative works of the Skyline/Geomni data are permissible if credit to the companies is included in the work. This means the data are equally useful in the public space, such as municipal hearings or on YouTube, as it is within the customer’s business.

Reality Modeling Challenges

I’ve described just one set of possibilities with the offerings from Skyline, Geomni, and their beneficial alliance. It’s difficult to find a more complete solution from a single set of vendors. Others exist in various forms, with their own advantages and disadvantages; with that in mind, let’s revisit the opening considerations.

Storage

Reality-capture raw data can be massive. An average 30-acre site can easily require 5 GB of imagery and 20 GB of laser-scanned data. The complexity of the site can increase these numbers dramatically. An active reality-capture team can easily create 5-10 TB of data per year. If airborne lidar, mobile lidar, and airborne imagery are involved, these numbers easily inflate to hundreds of terabytes.

Traditionally, this wealth of data is dumbed-down into simple CAD elements to serve an engineering market lagging behind current topographic technologies. The data also often get treated as a single-use case specific to an active project. When the conversion to CAD is complete, the data are archived to passive storage, saving active server space.

To fully realize the value of the data, they need to remain complete in detail. Reuse and re-purposing data from their original acquisition provides added value. Re-purposing usually occurs over time, requiring the data to be accessible far beyond the standard time frame of a single project.

This is one of the primary benefits of the mesh. It converts comprehensive 3D data to a more usable and intuitive format while saving memory by eliminating redundancy.

When considering a mesh solution, keep in mind that not all meshes are created equal. The grid that makes the mesh should scale based on the complexity of the surface. This means that flat, consistent surfaces will have a larger mesh; curved or irregular surfaces will have a finer mesh to better define the shape. A dynamic mesh helps reduce overall data size and storage needs. PhotoMesh is built on these principles.

There are many paths to storing massive amounts of data, and the best choice depends on the ultimate use of the data. If reuse and accessibility are important, a global hosted solution, such as the Skyline Globe Server, may be a wise option. A GIS platform like this encourages persistent use of the data. The SGS architecture provides the customer the flexibility of privately hosting the data or leveraging cloud hosting for scalability. Privacy, accessibility, and costs should all be considerations when exploring storage.

Accessibility

While smaller than the raw data, even a finalized mesh can be resource-demanding. To make this data more accessible to clients and mobile devices, the mesh should be delivered as needed. This requires streaming the data.

The SGS has attractive advantages. It’s built as a hosted data-streaming service. It may be placed inside the customer’s network for scalability and speed or outside the network to provide access by remote users and clients.

Many mesh streaming services are provided by third-party vendors, but they often require uploading the data to a remote cloud service. Depending on internet throughput, this can be a very time-consuming process. This can also become costly since hosting services typically scale the fees based on the amount of data being stored.

Security

There are ongoing efforts to make geospatial data open and accessible for everyone. While there are understandable benefits to open data, not all data is equal. You may need to restrict access to data due to sensitivity or proprietary investment.

Support for user-access rights is essential to control data access and editing. The most efficient means of providing security for the data is at the server; security based on user rights and group rights is a common feature in server architecture.

Once again, the SGS is built with security in mind. Whether it is a self-hosted server or a third-party data host, it is vital to understand how security and user permissions are managed. It is good to know these answers before a sophisticated client asks the questions.

Ownership

One of the biggest concerns in the reality-capture profession is ownership of the data. This is a legal issue and may require qualified counsel.

If you are a provider of reality capture services, your contracts should be written so you maintain ownership of the captured data. Make sure to include language that allows reuse of the data because some of the key benefits of reality capture are reuse and re-purposing.

There are times, particularly when working with government entities and municipalities, that the client already has language in their contract giving them exclusive ownership of the data. Be sure to have proper controls over the data to keep it isolated from access by unqualified users. The previously discussed security model is an important consideration.

Skyline’s business model provides a license to the SGS. As the customer, you get to choose how the SGS is hosted and subsequently have full control over the data ownership.

Unintended ownership problems can arise when using third-party hosting. It is vital to understand the End User License Agreement with the third-party host. It is not uncommon to find language that allows the host to access and use the customer’s data as they see fit. In practice, this might not be a great concern, but it could be abused and at the very least may be open to broad interpretation.

Depending on the contract with your client, you might also be violating your client’s rights by giving access rights to a third party. If the client’s contract has the previously mentioned exclusive ownership, you are definitely violating the contract. When dealing with an exclusive contract, leave the hosting considerations to the responsibility of the client.

Flexibility

A final consideration is the flexibility of the methods being employed. Reality capture and modeling often require a fusion of multiple sensors and quite possibly multiple sources.

PhotoMesh has already been offered as a comprehensive tool for creating a reality mesh from a wide range of sources, and the number of mesh-creation programs is continuously growing. Their capabilities and limitations are considerable. Research is important.

Depending on the scale of the project, it may be necessary to use varied sources of reality-capture data because data of varying accuracies and precision can still be valuable. These data may come from multiple sources, such as privately captured ground-based data, private aerial data, streaming data providers such as Skyline Atlas and Geomni, and even crowd-sourced data.

It takes a robust, 3D-oriented system to combine these varied data sources into actionable intelligence. TerraExplorer was built for this purpose. A GIS system is far more capable of managing a wide range of geospatial data than CAD; flexibility is the hallmark of GIS.

Reality capture and modeling are gaining momentum as organizations seek to maximize their data investments; reality models will soon be the ubiquitous method of representing the land for broad analysis or for engineering design.

Geospatial data have always had challenges. Reality modeling solves many of them. The three-dimensional, photo-realistic aspect of the technology is particularly useful for conveying a great deal of depth and context.

Reality modeling also creates new challenges. Fortunately, the technology is mature enough that comprehensive solutions are now available, Skyline providing one of the more robust options.

Topographic data-capture has entered the next phase. Are you ready to embrace the mesh?

Series Navigation

<< The BulletArcGIS Indoors >>