Working with geospatial data is, unfortunately, far from simple. Although, graphical user interfaces, data and analytics marketplaces, and driving consistency in access to different data sources is going some way to increase accessibility to an industry that has traditionally had a high barrier to entry.
As a customer support engineer at UP42, I support a fair few customers that encounter the same recurring issues. As such, I wanted to share these problems, along with my solutions to them, to help you - whether you are a geospatial newcomer or a seasoned GIS pro - avoid them in the future.
First, a little about me—before joining UP42, I worked as a technical support engineer for the photogrammetry company, Pix4D, also located here in Berlin. My focus was on troubleshooting geolocation issues, 3D modeling, and the overall accuracy of the software-generated point clouds. As a Python and MATLAB enthusiast, I reached the point where I wanted to get more hands-on with geospatial analytics. Thus, I arrived at UP42, where I now support customers like you in correctly accessing and analyzing images acquired by various sensors. I hope to impart some knowledge from my experience to help you avoid the common mistakes and perhaps even keep you out of my inbox!
1. Incorrect Parameters
Since I started working at UP42, I have noticed that one frequent issue lies in configuring the job parameters. The job parameters dictate how analysis should be conducted and are specific to both the data source and algorithms that users of UP42 are accessing. For example, when obtaining Pléiades imagery, users can specify a scene ID to ensure that they access the exact image they want from the archive. These parameters need to be adjusted in JSON, and with many of our users working solely in Python, issues sometimes arise...
Can you spot the mistake in these JSON job parameters?
{
"oneatlas-pleiades-aoiclipped:1": {
"ids": null,
"time": "2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00",
"limit": 1,
"zoom_level": 18,
"time_series": null,
"max_cloud_cover": 10,
"panchromatic_band": false,
"intersects": {
"type": "Polygon",
"coordinates": [
[
[
26.100103,
44.430401
],
[
26.105007,
44.431282
],
[
26.107497,
44.426839
],
[
26.101134,
44.426065
],
[
26.100103,
44.430401
]
]
]
}
},
"tiling:1": {
"nodata": null,
"tile_width": 768,
"tile_height": 768,
"match_extents": false,
"output_prefix": "",
"augmentation_factor": 1,
"discard_empty_tiles": true
},
"orbital_pleiades_cars:1": {}
}Invalid job parameters with the tile width and tile height set incorrectly
{
"oneatlas-pleiades-aoiclipped:1": {
"ids": null,
"time": "2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00",
"limit": 1,
"zoom_level": 18,
"time_series": null,
"max_cloud_cover": 10,
"panchromatic_band": false,
"intersects": {
"type": "Polygon",
"coordinates": [
[
[
26.100103,
44.430401
],
[
26.105007,
44.431282
],
[
26.107497,
44.426839
],
[
26.101134,
44.426065
],
[
26.100103,
44.430401
]
]
]
}
},
"tiling:1": {
"nodata": null,
"tile_width": 1232,
"tile_height": 1232,
"match_extents": true,
"output_prefix": "",
"augmentation_factor": 1,
"discard_empty_tiles": true
},
"orbital_pleiades_cars:1": {}
}Correct job parameters with the tile width and tile height set correctly in lines 40 to 42
Indeed, using UP42 requires some familiarity with JSON syntax so that each block gets correct values for each variable. With such various block types on the UP42 marketplace, from satellite imagery to object detection algorithms, it is impossible to derive a standardized set of job parameters that can be automatically applied or edited within the UI. I strongly recommend that our users consult the UP42 documentation and UP42 marketplace block descriptions before running any job. They can find examples of values they can insert in the JSON job parameters.
The UP42 marketplace shows block details as well as guidance on job parameters
2. Invalid AOI Geometry
Another frequently occurring issue is getting an invalid geometry error after uploading or drawing an area of interest (AOI). AOIs are usually polygons stored in vector files. To get an overall idea of what this means, vectors are composed of discrete geometric locations (x, y values) known as vertices that define the shape of the spatial object. These erroneous geometries are connected to broken conventions of vector files. Typical causes of an invalid geometry error are self-intersected vertices, duplicated coordinates, complex geometries, holes in polygons, lack of coordinate reference systems, and multipolygons.
Fortunately, various tools help to check the validity of an AOI. Users can either open and cross-check the AOI in the online GIS editor https://geojson.io/ or repair the geometry by applying functions such as Check validity and Fix geometries in the free GIS software QGIS. The example below shows how the function Check validity automatically detected the issue of the AOI.
Displaying an AOI with self-intersected vertices
Applying the Check validity function to detect the place where the self-intersection is present
3. Output Image Resolution
One tricky issue that I encountered during my work was to mediate the difference in expectations and reality regarding the visual quality of satellite images. The quality of satellite imagery depends upon many external variables outside of the control of UP42 or even of data providers. This risk should be known, especially by users who have a geospatial background or a broad understanding of sensor engineering. Rest assured that I am not asking anyone to have a Ph.D. in aerospace engineering. Still, I am trying to make users aware that satellite images are generally prone to various types of noise. This noise may be added to satellite images due to atmospheric noise (e.g., aerosols, clouds) or sensor noise (e.g., angle at which the satellite views the ground at any given time).
Another type of noise is the salt and pepper noise (also called speckle), usually generated from errors in data transmission or coherent backscattering (in the case of SAR images). To summarize, there are quite a few sources of noise. Some image artifacts could be corrected using standard pre-processing methods, such as atmospheric, radiometric, and geometric corrections. If this did not help, several more advanced approaches could clean the images from noise, such as linear or nonlinear filters, but these are usually rather cumbersome. UP42 also offers several super-resolution processing capabilities to automatically improve image resolution and drive consistency between the different bands in a satellite image.
Before and after the application of the UP42 Super-Resolution block
4. Job Runtime
One problem which sends users to us is the job runtime. Quite simply, regardless of the availability of cutting-edge infrastructure and ample GPU power, geospatial analytics with UP42 or otherwise is compute-intensive. And with many algorithms on UP42 being based on deep learning and many of our users running analytical workflows on vast AOIs, jobs can run for longer than expected.
To address this, UP42 is already planning to integrate new functionality that estimates and displays the job duration to you before you begin your analysis. Additionally, our Python SDK users can leverage parallelization - functionality to run traditionally concurrent analyses side-by-side - reducing the time you’ll spend twiddling your thumbs waiting for the results of your analysis!
import up42
up42.authenticate(project_id="12345", project_api_key="6889")
project = up42.initialize_project()
#Do the following step to be able to use the parallelization feature
project.update_project_settings(max_concurrent_jobs=10)
workflow = project.create_workflow("workflow_airports", use_existing=True)
selected_block = "sobloo-s2-l1c-aoiclipped"
workflow.add_workflow_tasks([selected_block])
workflow.get_workflow_tasks(basic=True)
#Below is how you would construct parameters and run jobs in parallel and finally downloading the results
input_parameters_list= workflow.construct_parameters_parallel(geometries=geom.geometry.to_list(),
interval_dates=[("2018-01-01","2020-12-31")],
geometry_operation="bbox")
#Running job in parallel
real_jobs = workflow.run_jobs_parallel(input_parameters_list=input_parameters_list)
real_jobs.download_results()
print("finished")Example of parallelization with the UP42 Python SDK
5. Archive Image Frequency
Many of our users hear phrases like "daily revisit rate" concerning constellations like Pléiades and get understandably excited! Indeed, the increasing frequency of high-resolution satellite image capture has unlocked a plethora of monitoring use cases. However, satellite revisit rates refer to what's possible when tasking an image, rather than guaranteeing that imagery is available for every area on Earth every day.
Searching the archive for Pléiades and SPOT imagery of Ernst-Reuter-Platz, Berlin
When imagery is available for a specific sensor, it depends on a range of factors, and sensors acquire images at various intervals from daily to once every 16 days or even less. It is hard to please everyone, so we recommend purchasing high-resolution satellite images acquired through custom tasking operations. The prices are affordable, and the terms of acquisition are transparent.
——————————————————
Working as a customer support engineer is a challenging kind of work. Still, it gives me an excellent overview of our users' experiences when exploring the UP42 platform and building geospatial products. Replicating user issues is one of the best ways to deal with errors, investigating potential causes, and diving more and more into remote sensing theory through daily practice. I do not get bored, and I learn every day, which makes things very exciting. I hope these lessons and tips that I've learned will help you sidestep some of the most common pitfalls when accessing and analyzing satellite imagery!
Whether you are just getting started with geospatial data or the UP42 platform; or you are already familiar with the core concepts and want to explore more advanced functionality, the UP42 documentation is a great resource to take your geospatial development to the next level.
