The door to open source geospatial software has been unlocked. It’s time to take a peek inside.

 

The open source geospatial movement is not purely about “free” (as in no cost). There truly are other aspects of “free,” as in “free to develop without constraints,” and free access to public geodata without necessarily having to invest in proprietary software.

OSG. We’ll use that acronym in this article for open source geospatial software, as there appears to be no universal acronym. OSG is not about “hacking.” And it is not always “free”; some are developed on contract via a growing industry of consultants and OSG “bridge” companies that provide value-added suites and support (see the sidebar “Lowering the Cost of Free Software.”) OSG is not just a bunch of nerds rebelling against the status quo; OSG has become a serious and viable set of tools that can complement the proprietary suites and that does not have to be adversarial.

OSG has become quite real and even institutional: states, federal agencies, and even whole countries have adopted “open source first” polices. A shining example is the Ordnance Survey of the UK. Read about it in “Open Source in the UK” by Nicholas Duggan in this issue.

OSG Drivers

A culture of customization: In any technology there is a drive to customize. Early CAD users discovered that AutoLisp and scripting skills were at first desired, then required skills in job postings, as was Avenue scripting for Arcview users and Python scripting for modern GIS users. Forbes noted in 2013 that coding/programming were among the top five skills to get you (and keep you) employed; there are only so many coders and limited corporate coding positions, but you can’t stop people from coding. Proprietary GISs embrace elements of this user-developer community, including software-developer networks and kits and application programmer interfaces (APIs) with varying degrees of “freeness” and “openness.”

Apps, Penguins, and the cloud: OSG grew from the same open source movement as the Linux operating system (with its Penguin mascot). Facing code-licensing restrictions, the growing independent coding communities embraced open source. People are comfortable with small-source products; think of the millions of apps for that mobile-in-everyone’s-pocket, and comfort with the cloud also fosters willingness to try open source.

Access to public data: The increasingly tech-savvy “geo-democratized” public is aware of huge pools of digital data ready to be tapped and want non-proprietary formats. But also note that commercial GIS companies are big promoters of open GIS data and standards.

Big Data: More than 80% of the world’s data has been generated in the past two years. We live in a mass data capture world: million-shot-per-second lidar, all the way to cube satellites. We have more data and more uses, and some people feel that legacy development models do not move fast enough.

Lowering the Cost of Free Software

Just because core software is free does not mean there will be no costs associated with deployment and operations, especially if implemented enterprise-wide. Many of the costs would be no different than if commercial software were purchased: there is training, IT costs, customization (if needed), and ongoing support.

A growing number of OSG “helping hand” or “bridge” software development firms and consultants have formed, such as Faunalia, Astun Technologies, Boundless, and more. Many of these teams continue to be directly involved in development of core OSG elements. The idea, as some have put it, is to lower the cost of deploying free software.

One example of how such firms grew organically from the open source movement is Boundless. This company came out of OpenPlans, a non-profit that for more than a decade helps folks plan for more livable cities: more pedestrian and bike alternatives, fewer cars, better traffic, more transit options.

OpenGeo (now called Boundless) was a geospatial division of OpenPlans. Chris Holmes was lead developer of GeoServer, a java-based open source server to make it easier for urban planners and transportation planners to share and edit transportation data. The geospatial team at OpenPlans did a lot of development in interoperable web services and worked closely with OGC. In many ways they were “in the cloud” before people in commercial GIS started talking about it.

GeoServer grew into one of the foundations of the OpenGeo Suite, which Boundless offers commercially to help people implement open source tools, like QGIS, PostGIS, etc. The team was encouraged to spin off. OpenPlans had a specific non-profit mission in urban planning: the development environment was limited, and going commercial would support broader R&D. The types of clients range from federal (NOAA, USGS), defense and intelligence agencies, states and municipalities (Mass GIS, Howard County, MD), utilities (San Jose Water), international (Ordnance Survey UK, World Bank), and commercial (Ericcson, Boeing, Lockheed Martin).

We spoke with some of these “bridge” developers and consultants. To summarize their philosophy (and that of OSG in general), they asked, “What does it profit to collect a lot of data, just to have it go away after you’ve styled a great map?” OSG (and GIS) have always been about way more than just the picture. If you just want to make a great map and share it, that’s fine, but if you want to share the data in an API (application programmer interface) or a service that someone else can operate on, make their own maps, analyze, and segment the data, then make that available for others—all the better. An underlying mission it seems of OSG users and what they task consultants to do is to build open, collaborative, and interoperable spatial IT, on their terms, and not someone else’s.

 How OSG Works

You download components, install, self-teach, get a book, take a class, do tutorials, go online for help. Learning and implementing take time, so there are incidental costs, even if the software is free.

The differences in OSG and proprietary geospatial software can be more pronounced in the areas of customization and development. With proprietary, there are, of course, licensing and sharing restrictions; with OSG there are fewer. To an open source idealists, the only limitation is your imagination: you can do your own coding, seek peer coders, collaborate via web portals, hire a consultant, and, depending on licensing, freely distribute this to others.

Foundation Blocks of OSG

From the outside, OSG looks like a ballpit full of little kids thrashing about (as one OSG insider put it). But there is method to the madness, as well as several popular database types, server and desktop software packages, code families, non-profit organizations—you can build nearly anything you wish with them. Here are some of the many foundational
elements:

• OGC, the Open Geospatial Consortium: OGC is a standards’ consortium (and more) of nearly 500 private public, private, and academic entities.
• PostgreSQL: open source object-based database. One of the most popular tools for the OSG community, it provides extensible power for the increasingly large data sets: postgresql.org.
• PostGIS: adding the geospatial component to PostgreSQL. Support for points, lines, polygons, geometric collections, and more: postgis.org.
• GDAL, Geospatial Data Abstraction Library: a widely adopted open source library of raster and vector data format exchange types: gdal.org.
• OSGEO, the Open Source Geospatial Foundation: a leading organization promoting OSG development. Their key conferences are called FOSS4G (free and open source for geospatial): osgeo.org.
•  GeoServer: an open source server system that consumes geospatial data in nearly all formats and serves them onto the web: geoserver.org.
• Python: around since the early 1990s, the main query language for most current GIS like QGIS and ArcGIS.
• GeoJson: an example of simple foundational code for addition of nearly all spatial data types to data: geojson.org.
• QGIS, Quantum GIS: prevailing OSG desktop GIS alternative. QGIS was begun in 2002 by Gary Sherman to solve a problem; he needed to be able to view and analyze a big pile of data: qgis.org.
• GitHub: a collaborative web-based service that offers development teams the ability to work in a completely distributed environment, globally, and in real-time: github.com.
• Others: OpenLayers Leaflet, MapServer Geoforge, GeoDjango, Safe Software FME, and CartoDB, C++, GEOS, SQLite, GRASS GIS, and many, many more.

Look them up. Not all of these are purely open source, but they are some of the essential tools for development and use of OSG.

No Free Lunch

There are fans and detractors, plus several potential downsides to implementing OSG that get aired regularly, more germane in years past than now. The following folks are not necessarily detractors, but we asked them to echo common concerns they hear.

There’s the common concern of “lack of single source of tech support/accountability,” says geospatial industry watcher and editor of the Geospatial Solutions e-zine, Eric Gakstatter. “It’s great if you have the in-house capability, but it can be tough if it’s a single person or small GIS group.” He adds, “lack of a strategic product development road map.”

Jonathan Murphy, managing director of GoGeomatics, the popular geomatics magazine forum and geo-community hub based in Canada (gogeomatics.ca) offers seminars for businesses and consultants weighing the pros and cons. A concern is “help files and documentation,” says Murphy. “Sometimes these are nonexistent or fairly sketchy.” He adds, “technical support: if you do get a hold of someone, they might be working at a level the user is not.”

Murphy holds cautious optimism for the future of OSG: “Open source is great, and it does have a complementary role to play in the sector. Why buy another license for a very particular processing requirement that is not part of your regular work flow when you can use an open source tool and save yourself the bucks?” Though Murphy concludes, “Open source could benefit from more time to gain a higher level of trust.”

 

An Interview with Anita Graser

To get a better sense of the prevailing schools of thought among the OSG developers and users, we interviewed Anita Graser, whom many hail

as one of the OSG rising stars and one of the new faces of GIS. Author of the popular Using QGIS 2.0 guidebook, OSG personality, blogger, author, and speaker, Anita is a researcher with the Austrian Institute of Technology.

Q. On the surface it might look like the appeal of OSG is “free,” but how would you characterize the deeper benefits?

AG. For example, as a student my main interest was to have a GIS tool that I could use outside the university laboratory and for internships. As a researcher, it is essential for me to be able to “look under the hood.” As a software developer, I appreciate the collaborative development environment. Anyone can pick up existing functionality and improve it for the benefit of the whole community instead of having to wait for a commercial provider to react to a request, an important point for industry and government users of OSG.

Is QGIS the tool that is bringing OSG to the masses?

Over the last couple of years, QGIS has become the leading OSG desktop GIS; for many long-time desktop GIS users it is the first tool from the OSG ecosystem that they will give a try. For quite a long time, OSG desktop GIS projects were playing catch-up with commercial GIS offerings, but now there are realms where OSG is actually leading, e.g. OpenLayers, UMN MapServer, GeoServer, etc.

Last year, when we celebrated the release of QGIS 2.0, I published “Learning QGIS,” a quick introduction to QGIS and soon a new version for QGIS 2.6. [Look up the title on packtpub.com.] I think we have reached a point where QGIS actually offers a better solution for some use cases. E.g. cartographers will appreciate the newly added color-management capabilities and advanced styling options.

What does a typical OSG development workflow look like?

When we look at the development workflows in [for example] QGIS projects, we can distinguish between two main categories: development of the core QGIS application (usually in C++) and development of plugins (usually in Python). Most new developers today start out with writing a plugin with just an idea or use case. In case you need development support, the project provides a list of commercial providers. If you want to share your plugin, you can do so on the QGIS plugin repository, provided you fulfill minimum requirements. If a plugin proves extremely useful for a big part of the user base, it can be application.

There is also the core QGIS application which is maintained mainly by a group of so-called “core committers” (developers with rights to add new code to the application). But since the QGIS project hosts the source code on GitHub, anyone who wants to contribute can do so by providing improvements using the “pull request” feature, which is a collection of code changes reviewed by the core committers and included into the application if they approve it. Decisions are typically made in a democratic process, or if there is no consensus by the project steering committee, which I am part of.

Commonly stated concerns about using OSG are support and interoperability, but now we see widespread adoption even by national governments. Why do you think the comfort level has increased?

While a lot of people were skeptical, they cannot ignore the fact that a growing number of early and not-so-early adopters are already successfully using open source solutions. In most cases, big organizations such as governmental organizations won’t suddenly switch to open source. Instead, they often introduce open source as an alternative so they can get GIS tools to more users than they could cover with their commercial licenses. More universities are now providing training using both commercial and OSG tools.

Commercial GIS and OSG are sometimes seen as adversarial. Should they be viewed more as complementary?

I think there is a case for both commercial and OSG GIS, and in most settings they can coexist side-by-side. Interoperability between different tools, therefore, is a key factor. That’s why I am a big supporter of open data formats and recommend their use over vendor-specific closed formats.

I find more people who use GIS that are not employed in legacy style “GIS Departments” or identify themselves solely as “GIS.” Has the landscape changed for GIS and its practitioners?

I like to think of myself as a Geographic Information Scientist using a tool set, including GIS, to target research questions and project tasks. I consider myself very lucky to be in a position that allows me to continuously explore new ideas and approaches. In general, I see it as a positive move that OSG and GIS are becoming more integrated—and maybe more mainstream.

 

The Open Question

As we note in this month’s editorial, OSG is too “real” to overlook, and there appears to be enough demand for both commercial and open source solutions. The door is open. Maybe now is good time to take a peek inside.

 

Paul Ramsey is a programmer, consultant, and visionary who advocates for open source (of all kinds) and has become one of the key personalities and evangelists for OSG. Here Ramsey speaks with us of the early days of open source.

Software development is an industry of the mind, but some basic tools are needed. Startups with brilliantly innovative ideas struggled under the weight of initial investment in proprietary operating systems, office software, and geospatial software. Open source had a certain appeal. Ramsey founded the PostGIS geospatial database project in 2001 (with an elephant logo) and has continued consulting for the company Boundless and others.

“Freedom to redistribute, freedom to modify, and freedom to redistribute your modifications” is how Ramsey summarizes some of the goals of open source. He notes that the internet itself is “one of the earlier examples of commons-based peer production. The first 100 nodes of the ARPANET (Advanced Research Projects Agency Network) went to computer science departments’ first virtual community.”

Ramsey gives another example of such peer-based development: “Wikipedia works basically the same manner as open source except there is the one central copy of the information. The principles are still there.”

With open source, says Ramsey, “Anyone can take a copy, give it to anyone else, put it on your computer, device, take your copy, change it—all done under creative commons [type] licenses. Another example of this kind of mixing in other cultural domains is rap music: full of other kinds of intellectual property mixed into new forms, both the old and the new, and that is how culture grows.”

“Remixing is not disruptive code,” says Ramsey, “but methodologies of doing it are. We have examples piloted in open source manner like in Wikipedia, and in the geospatial world, OpenStreetMap (OSM).” OSM is a peer developed (mainly, but not purely “crowd sourced”) map database of the world. It is a common data layer option offered in even some proprietary GIS.

OSG consultants offer a different business model than proprietary geospatial. As Ramsey puts it, “The proprietary software model economically is ‘Pay us money for our last year’s old code,’ and the open source software business model is ‘Pay us money and we’ll write new code.’”

Or, as he further illustrates: “The proprietary model is that ‘We’ve got this fence around a big pile of something that has value, or we try to convince you that it does, and you pay us money and we will open the gate.’ The open source model is ‘Here is a whole bunch of cool stuff with no fence around it, and if you want the bunch of cool stuff to get larger you can [develop it yourself] or pay someone to make it bigger.’”

We asked what hurdles there are to wider adoption of OSG. Certainly not technology, he notes, but more about existing structures: “The tech will change underneath, and those who are somehow preventing institutional change may not realize it is changing all around them. How many types of measurement are there that are instant and available for just about anyone to go ‘Okay, here I am, or here this is’?”

Noting the combined power of big open geodata and improved, instant, and accessible positioning, Ramsey concludes, “The fact you can put all of these together: this further strengthens the idea of open source and independent data gathering and in-housing mapping rather than big contracts.”

Above map picture: Map contains Ordnance Survey data @ Crown copyright and database right 2014

Series Navigation

<< Open Source in the UKAll Over the Map >>