This post is dedicated to a good friend, Chris Johnson, who, when we worked together a few years back, introduced me to this cool new word: VIRTUALIZATION. You see, Chris is an IT guru (see: enterprise systems architect)—a friend that every GIS guru should have so that you’ve got someone to pick up the pieces when you’re throwing your project out of the window. In the five or six years we worked together, I threw lots, mostly due to having to deal with extremely large data and very slow computation.
At this point I need to explain that I was using an HP Z600 workstation with tons of Xeon processors, a gaming graphics card, and 64GB+ of RAM (we had already identified Esri’s contiguous RAM requirements). The machine should have been a solution for everything, but as soon as I got survey data in, usually 1TB of xyz data, I’d go to chop it up or interpolate to raster and be waiting a week. A WEEK! When you’re trying to justify your worth at a job and costing the company approximately £4500 a week just for converting some geospatial data, it made for some interesting annual reviews.
A server serves “virtual computers.”
“It’s not me, it’s the hardware.”
So, the day came when Chris and I had a chat. It’s always the IT guy’s fault, right? Well, Chris asked why I was running the processes on my desktop when the company had a Windows 2012 server with hundreds of cores and RAM. He might be able to virtualize the software so that the server could work as the hardware to churn through the data.
Now, all these years on, it doesn’t seem so crazy. After all, almost all the top survey companies like Trimble, Hexagon, and Topcon have a cloud-based engine. What is this cloud-based engine? Yup, you guessed it, it is essentially a server or bank of servers that just “churn” through data. Processing the data in this manner is typically called, “cloud processing.”
Let’s have a think about what a modern workplace server is. A few decades ago they were systems in place to serve as an internal email and storage facility. Recently, company servers have developed substantially. Just looking at the Windows 2016 Server operating system makes your mind boggle; it’s more computer OS (Operating System) than server OS. The GUI (Graphical User Interface) opens the same as the desktop counterpart, and all the applications reside under the Windows icon. It’s scarily similar; in fact I’ve been told that the server and desktop OS share most of their infrastructure.
The Server 2016 Interface
The Magic of Desktop Virtualization
If you’ve got this far, I am assuming you get the idea: using all those cores and RAM makes it go faster, which is great for processing. This is only part of the story.
Imagine being able to log into any computer, anywhere, and seeing your desktop exactly as you would on your home machine. All your software is there and works just as quickly (if not quicker). Recently I was being shown the familiar windows desktop with Microsoft Office from an Apple iPhone—I kid you not.
Where all the “grunt” is done on the server, the GUI only needs to send and receive the small changes visually that occur from the interactions. This means that you can quite easily perform those mammoth ArcGIS, QGIS, or Pix4D jobs from your phone (providing there’s a signal).
Why aren’t you using it right now? Simply put: cost. On a small scale, although the price of servers and rack space has come down a lot, there is still a price to be paid. Unless you are processing a lot of data, it might just be better to wait those extra couple of days for the odd job here or there.
Many servers make light work.
But don’t despair yet. A few years ago cloud processing was a thing of dreams, and now the likes of Trimble, Leica, Topcon provide it as part of their offering. Although many large businesses may run a basic virtualized environment, a lot more of the medium and smaller-sized businesses are virtualizing their systems due to how cost-effective it can be on licenses, downtime, and time lost to processing.
How long will it be until virtual desktops with GIS processing software like QGIS, Meshlab, ArcGIS and Pix4D are available to rent? I don’t think it will be long!
Try these resources:
- White Paper on Virtualisation of ArcGIS Pro from 2015
- Desktop Virtualization (using Server 2016)
More on virtualization from providers:
