Cubesat Imager - From Data to Foresight in 7 Steps
Thys Cronje
Chief Commercial Officer at Simera Sense
The amount of Earth Observation satellites launched is increasing exponentially daily, and CubeSat imagers for Earth Observation play a significant role in this expansion. However, the real challenge is to transform the data generated by CubeSat imagers into actionable foresight rapidly. Valuable and actionable information, at high accuracy, needs to be extracted and made available to decision-makers to solve real-world problems.
The CubeSat Imager Paradigm
Conventional paradigms are not working for CubeSat Imagers. The truth is there is no shortage of advanced image processing techniques and platforms, but the old-school data-driven approach does not fit nanosatellites that well.
Nanosatellites are mass, volume, and power constraint. These constraints are creating several bottlenecks for the satellite provider and operator. Nevertheless, it is still capable of capturing terra bits of Earth Observation data daily.
For example, the products within the xScape100 and xScape200 ranges can create and store up to 1 Tbit of data daily. Downloading all of these data with a conventional 50 Mb/s X-Band downlink may take about 350 minutes, just below six hours. When considering the limited ground station interaction time the downloading of data may take days.
Frequent access to X-Band ground stations is expensive. The downlink cost can quickly run up to €0.1/MB or more if high priority and high uptime links are required. With downlink cost calculated as a function of the total throughput, increasing the downlink bandwidth may partially address the downlink time issue, but you may not address the cost.
From Data to Foresight with CubeSat Imagers
Creating advanced onboard insight is an alternative to addressing the downlink bottlenecks and the associated costs. The power of onboard computers and dedicated processors is increasing rapidly. These systems are usually part of a larger framework that allows the rapid deployment of onboard applications.
Transforming RAW data from a sensor into actionable insight and foresight requires quite a few steps. In particular, these steps involve detecting, recognizing, and identifying features, objects, and targets. By elevating the data to such a level of information, one can quantify specific situations to predict trends.
Incorporating this level of processing within a CubeSat imager is easier said than done. However, artificial intelligence and machine learning, especially deep learning neural networks, are advancing fast. Using these algorithms is becoming standard practice to extract feature images onboard nanosatellites reliably and quickly. Furthermore, with a deep understanding of image objects and the related domains, one can classify image features and make predictions with high accuracy by incorporating contextual geospatial layers.
#1 RAW Earth Observation Data
In an earth observation context, we see RAW data as the individual, digitized pixels within an image, where a pixel is the smallest data unit produced by an optical payload’s sensor. In its purest form, Earth Observation data is a measurement of a reflected or transmitted signal. For a passive optical payload, reflected light from the earth’s surface is detected and transformed into a digital stream, and during a calibration process, a unit is assigned to the pixel value.
Earth Observation analysts frequently classify data in terms of spatial, spectral, radiometric, and temporal resolution. These parameters are crucial in determining the data’s information content to detect a specific target. In this case, detection means that the optical payload can “see” the target or object at a specific resolution, contrast, and spectral content.
The real value of CubeSat imager data is unlocked during the data correction and calibration process. However, incorrect geolocation, spectral band misalignment, and pixel intensity variations drastically impact the performance of the Earth Observation data. Therefore, during the calibration process, geometric and radiometric variations across the image plane and the data set are corrected.
Due to size, weight, and power constraints, a CubeSat imager frequently lacks onboard calibration capabilities. Consequently, vicarious calibration techniques, which use known targets on the earth, are used to calibrate CubeSat imagers and intercalibrate between missions.
Hence, when referring to Earth Observation data, it is essential to mention the calibration level. Without a proper calibration step, it becomes challenging to transform the data into information and compare various data sets.
#2 Recognizing features is the first step to unlock information
When organizing data so that we can recognize features, information is created. During this step, we recognize specific attributes and derive informative values. Here, recognition does not mean that we can identify specifics regarding the object but rather “index” the detected object into a category, like cars, rivers, houses, mountains, vegetation, roads, lakes, etc. The rule of thumb is, the object needs to be visible on at least two pixels to be recognizable and separated from another object by one or more pixels.
Per definition, Earth Observation instruments gather information to monitor and assess our environment’s physical, chemical, and biological content. To recognize specific chemical or biological features within an object, we need to include a range of spectral content into the data. For example, high spatial resolution within the visible range may not be enough to recognize patches of vegetation within a desert or discriminate trees from open land or other types of vegetation.
Knowing the type of features you would like to recognize influences the type of CubeSat imager one selects for a mission.
#3 Bringing Context to an Identified Object
When adding context to the information by identifying structures and objects, one is going beyond only recognizing and indexing, probing the difference between objects of the same class. An example may be to differentiate between crops, trees or to identify a specific river or highway. Therefore, when identifying specific objects, like an airport with specific airplane types, valuable context is added to the information.
To expand on the example above, when an algorithm onboard a CubeSat imager recognizes an airport by looking for runways, specific airplane types and various objects may be identified to provide context regarding possible activities, especially economic activities.
The same is true when identifying specific kind of crops and the area it covers. This additional layer of information provides immediate context for crop estimates to predict food security or to monitor change within the near future.
#4 Creating Awareness with CubeSat Imagers
However, by only identifying a specific crop and the area covered, the state of the crops and predicted yield may still be unknown. What one misses is the specific characteristics and, ultimately, the state of the identified crop. The same is true for the airport example: is the airplane parked at a terminal, hanger, or taxiing? Therefore, classifying objects relative to a current state unlocks the next layer of information and awareness for a specific situation at that specific point in time.
With 2m to 5m GSD in multiple spectral bands, CubeSat imagers can be extremely useful to monitor large areas, identify specific structures, and classify them according to their current state. Such an awareness of a specific situation is also vital to enable a tip and cue strategy for a cubesat constellation.
#5 Quantifying an Impact Creates Knowledge
As soon as classified objects and situational awareness are quantified, decision-makers can determine the magnitude of an event. Quantification is more than just counting objects; it encompasses the prediction of economic activities and socio-economic impacts.
In Earth Observation, quantification could mean various things, but in essence, it is the act of counting or measuring objects, areas, or events. For example, measuring the area covered by a specific type of crop or trees, the number of ships in a harbor, the area impact of a natural disaster, or the number of photo-voltaic units within a particular area. Knowing these numbers adds another level of information to situational awareness.
The ideal situation is to task CubeSat imagers to perform specific quantification tasks of classified objects and stream this data to decision-makers, eliminating the need to download a complete data sets.
#6 CubeSat Imager, the Ultimate Intelligent Agent
As humans, we define intelligence as the ability to understand complex ideas, adapt effectively to the environment, learn from experience, engage in reasoning, and overcome obstacles. For some, the idea to deploy this level of intelligence onboard a CubeSat imager may sound a bit far-fetched, but there is no reason why nanosatellites can’t act as intelligent agents. In this case, the intelligent agent perceives the environment under surveillance and maximizes its success rate. Therefore, the system needs to correctly interpret data, learn from such data, and use those learnings to create specific outcomes, all through flexible adaptation.
Take, for example, the monitoring of natural catastrophes like wildfires. First of all, a CubeSat imager can detect, recognize, and identify fires. The next step is to classify the fire and create a level of awareness, quantify the situation according to size and the close vicinity of infrastructure. The next level of information could be to classify the specific types of infrastructure in danger. Rapidly shared, situational information of this level is invaluable to first responders, and one can measure the intelligent agent’s success rate against the accuracy and timely availability of the information. Above all, there is no reason why this level of intelligence can’t be deployed onboard nanosatellites.
#7 Insight to Gain Foresight
We all understand the concept of a sixth sense, that ability to understand something instinctively, without the need for reasoning. Some may call it intuition, and it develops with experience. It requires the capacity to gain an accurate and deep understanding of a situation, better known as insight.
However, as soon as one can extrapolate a current situation to possible future outcomes, foresight is created. It is the ability to know where the fire is going, how fast, and the possible impact and outcome and direct the information to the appropriate decision-makers. Although to reach such a level of insight and create accurate foresight, information from different sources is required and needs to be fused with surveillance data from various sources.