Scalable is a fairly overused word in tech. It’s everywhere. If you go to nearly any software-as-a-service company’s site, you’ll see it. If you’re on LinkedIn, you’ll see it. X? You’ll see it.
We may be at the point where readers are starting to tune it out. Are there any SaaS products out there that aren’t scalable? If it’s not, perhaps it’s not worth buying, especially for bigger organizations. Scalability should be a given by now.
So why are we still going on about it?
Because, while SaaS is a relatively recent concept in the era of human business, Earth observation data platforms are even more recent. We thought it would be worth diving into the key elements of a scalable geospatial platform.
First we’ll talk about scalability in the general sense, and then dive into how platforms can solve its biggest challenges.
The definition of "scalability"
"Scalability" is business language. It's not a term that comes up in everyday conversation. Here's the definition, taken from Investopedia, which is probably one of the best sources for business language.
"Scalability refers to the ability of an organization or a system such as a computer network to perform well under an increased or expanding workload. A system that scales well will be able to maintain or increase its level of performance even as it's tested by growing operational demands."
When we apply this definition to a SaaS product, it should cover two different possibilities:
-
When a single customer uses the product, they're able to ramp up their usage of the product without a problem. This could mean more database storage, more operations per day, or any other number of possibilities.
-
As the product gains more customers, existing customers don't experience a decline in performance.
In one sentence: customers should be able to scale up their operations with the product without running into obstacles, and the product should be able to scale up its own operations to match an increasing customer base.
Scalability as it relates to Earth observation
When we think of a successfully scaled business, there are some obvious ones. Amazon, for example, started off selling books and now sells almost everything. Most big tech companies can be seen as successful at scaling, from Microsoft and Apple to Netflix.
But what does successfully scaling in EO look like? The industry is currently filled with big and small players. The bigger players usually focus on their own products. Smaller companies and startups are trying to fill the need gaps around buying different types of data from different suppliers as easily as possible. Like us.
And to do so, you need a platform. As we see it, here are the top 5 key elements to making a geospatial platform scalable, their challenges, and our solutions.
1. Efficient data ordering and management
It should be as easy as possible for individuals and small businesses to check out available EO data, compare options, and order.
But if users are doing this in a browser, it's not exactly scalable, especially when they need to place hundreds or thousands of orders. They also need a way to sort through and, preferably, classify their orders so that they can keep track of them.
How we're solving this
While our console is available to users who want to visualize their scenes on a map, users can then scale up their ordering via our API or Python SDK so that they don't have to enter every order manually. Instant price estimates show up as soon as AOIs are finalized, which helps with budgeting and planning. Users can save their AOIs and reuse AOIs across shared accounts. They can also use the same set of ordering parameters to quickly integrate new products into their workflows.
The UP42 platform lets users add custom attributes to orders, like titles and tags, so that you can easily sort and filter your orders, for individual or enterprise accounts.
2. Data standardization
One major challenge in our industry is the complexity of the data. Optical sensors collect very different types of data from SAR sensors. And even a single image may span tens or hundreds of kilometers and contain multiple layers of information in the form of different electromagnetic wavelengths. In addition to this, different providers often use different file types and formats.
All of this adds up to a lot of work for end users, who need to be familiar with many different types of data and know how to work with them.
How we're solving this
UP42 uses STAC (SpatioTemporal Asset Catalog), a specification that was designed to establish a standard in geospatial data. It allows searching across multiple providers for geospatial assets which share a common structure and set of metadata.
All data deliveries in the UP42 platform are transformed: raster data into cloud-native GeoTIFFs, and vector data into GeoJSONs. All transformed assets use the STAC specification, so you can easily search for, access, and stream them, without needing to worry about how to work with different data formats.
You can read more about how we're using STAC here.
3. Processing
As mentioned above, EO data is incredibly complex. Once the data is collected, it needs to be processed before it can be useful. What we refer to as "noise" often interferes with image collection, and can be caused by anything from temperature fluctuation and atmospheric effects to changing light levels.
While some users will know how to process data to get the best results from it, most won't. That's where we come in.
How we're solving this
UP42 offers pre-processing for the vast majority of our data products. For example, many optical imagery collections offer the following levels of pre-processing:
-
Primary processing makes an optical image suitable for generating digital elevation models, but still stays close to the rawest form of the data.
-
Georectification adjusts imagery to align with an existing coordinate reference system, which allows for more meaningful geospatial analysis.
-
Orthorectification is more advanced, and involves imagery first being georectified, and then orthorectified: distortions such as terrain variations, sensor angle, and many other factors are removed. This allows for accurate measurements and comparisons in geographic analysis.
We're also broadening our processing abilities even further with the addition of our pansharpening capabilities, which allow users to fuse color images (higher spectral resolution, lower spatial resolution) with panchromatic images (grayscale images with low spectral resolution but higher spatial resolution) for full-color, higher spatial resolution images. Expect even more processing updates soon.
4. Ecosystems integration
There are a bunch of popular geospatial/EO tools out there. If you've got a favorite, then you'll need to download the data from whichever platform you're using and add it to your tool. Sometimes this goes smoothly. Sometimes, it doesn't. At the risk of repeating ourselves, the data is complicated and things can often go wrong.
How we're solving this
ArcGIS Pro is one of Esri's flagship products, and sees wide usage across the industry. The UP42 ArcGIS Pro Add-in allows Esri users to connect their accounts to the UP42 platform, so that they can search for, order, and process imagery directly with ArcGIS Pro.
You can learn more about the add-in and download it here.
5. Data privacy and security
If a platform is truly scalable, then any size organization will be able to use it. Big companies usually have fairly strict data privacy and security policies, as well as organizational needs around account management.
EO data is extremely valuable, and needs to be protected from bad actors. So EO data platforms have a responsibility to keep their customer data safe.
How we're solving this
Earlier this year, UP42 achieved SOC 2 Type 2 certification.
Developed by the American Institute of CPAs (AICPA), SOC 2 defines privacy and security criteria for managing customer data in a service organization. SOC 2 provides detailed information and assurance by examining main data management aspects such as security, availability, processing integrity, confidentiality, and privacy.
For example, security refers to protection against unauthorized access such as theft or alteration of data. Availability means that system, products, or services are accessible for operation and use as per contract or service level agreement (SLA). Whereas confidentiality ensures that confidential information is protected and access is restricted to a given number of people in the organization that need such access for conducting their work. This could be, for example, pricing, discounts, personal information, IP address, and more.
A SOC 2 Type 1 audit assesses a company's cybersecurity controls at a single point in time. It will assess that a company has the systems and tools in place to protect customer data. However, it will not assess how effective they are. This audit is less time-intensive than a SOC 2 Type 2 audit.
With SOC 2 Type 2, the auditors will test how effective those processes, systems, and tools are initially over a couple of months, usually a quarter. The assessment is much more comprehensive and demonstrates the reliability of a company's systems. But getting the SOC 2 Type 2 certification is just the beginning. Now you need to reassure customers that getting certified was the first step and that you're immediately following up with an audit that spans a full year, so that there's no period without being SOC 2 Type 2 compliant, meaning you're always under audit. This is what we're doing at UP42: we conceive of our SOC 2 certification as an ongoing effort that makes sure we always comply with all the SOC 2 controls, as attested by our auditors. Our customers' concerns around cybersecurity are always on our mind.
In short: you can be sure that your data is safe with UP42. You can find more information on the different certifications here. You can also check out our trust center for more details on our certification.
EO as an industry: Nowhere near our full potential
If nothing else, we hope this article has at least helped you look at the word "scalability" in a new light. There's a reason that it's such a popular word.
But we hope you're taking away more than just that. EO is a young industry, and it's no wonder that we're facing scaling challenges: we have unique problems, and vast ambitions of capturing and analyzing titanic amounts of data on a planet-wide scale. Honestly, it's amazing that we've come so far in such a relatively short amount of time. Though there's still a long way to go.
Whether you need a few scenes, or want to scale up your EO operations, UP42 is here. Reach out and tell us about your use case. We'd be happy to help.
