1. Pelé and Garrincha: a fusion of talent
A partnership so effective that Brazil never lost a match with both in the starting lineup, Pelé and Garrincha rank among the most legendary forwards in world football. Pelé provided explosive power, accurate finishing, and overall technical excellence, while Garrincha offered unpredictable dribbling and creative play from the wing. Together, they presented a dual attacking threat that was difficult to contain.
Why did they complement each other?
- Pelé’s simple and efficient approach complemented Garrincha’s complicated and erratic dribbling abilities.
- Defensive structures could not simultaneously counter both players, resulting in persistent tactical imbalances.
- Key achievements: the duo maintained a 40-match unbeaten streak for Brazil when appearing together.
- Both players were critical to Brazil’s World Cup achievements in the late 1950s and early 1960s. When Pelé was injured in 1962, Garrincha led the team to the title, exemplifying the strategic significance of their partnership.
2. Fusing the EO stack
The success of an Earth observation strategy frequently relies on effective partnerships, because no single sensor is a silver bullet. Earth observation (EO) is moving beyond single-sensor imagery toward multi-physics intelligence: a fusion of data layers that record different physical phenomena to create a richer, more actionable view of the world.
Fusing three main modalities: Synthetic Aperture Radar (SAR), Thermal Infrared (TIR), and Hyperspectral Imaging (HSI), unlocks unique benefits. SAR reveals structure and moisture, thermal captures heat patterns and energy balance, and hyperspectral decodes the makeup of substances and subtle biochemical states. When fused, they create new, synergistic insights that are not possible from any single sensor, enabling faster, more confident decisions across weather, time, and terrain.
This blog explains what a multi-physics EO stack is, why it matters, how to architect it, and where it creates the biggest impact.
3. Why fuse SAR, thermal, and hyperspectral?
Each modality illuminates a different dimension of reality:
- SAR (X/C/L bands): active microwave sensing penetrates clouds and operates day or night. It is excellent for measuring surface roughness, dielectric properties (e.g. soil moisture), structure, and small displacements via interferometry (InSAR). It is the backbone for reliable coverage in all conditions and for detecting change, even under cloud decks.
- Thermal infrared (longwave/midwave): measures radiance and temperature, enabling information on evapotranspiration, city heat islands, industrial process heat, wildfire intensity, and thermal anomalies such as leaks, venting, or cooling. Thermal provides the energy component often missed by visible or near-IR sensors.
- Hyperspectral (VNIR/SWIR): delivers continuous spectral details across hundreds of bands. It distinguishes materials (minerals, plastics, vegetation species), chemical states (hydration, protein content), and stress (nutrient deficiency, disease, contamination) with high specificity.
No single sensor covers all operational cases. SAR brings all-weather cadence, thermal gives essential energy context, and hyperspectral explains what is present and its condition. Fused together, these modalities resolve complex questions such as what has changed, why it is changed, and what action should be taken. This greatly improves precision, recall, and confidence for downstream analytics on tasks ranging from environmental compliance to asset monitoring.
4. The multi-physics EO stack: an architecture blueprint
Designing a multi-physics stack means solving to achieve coherence, scale, and latency across disparate data sources. A practical architecture looks like this:
A. Sensing and tasking
- Orbits and constellations: a mix of SAR (for all-weather baseline) with targeted tasking of thermal and hyperspectral passes for enrichment. Consider revisit trade-offs: SAR for frequent monitoring, HSI/TIR for less frequent but high-information updates.
- On-board preprocessing: radiometric calibration, compression, and, where possible, rough cloud or quality screening to reduce downlink burden.
B. Ground ingest and harmonisation
- Geometric co-registration: exact alignment (sub-pixel where possible) across SAR, TIR, and HSI. DEMs and tie-point matching reduce parallax and terrain-induced errors.
- Radiometric and spectral calibration: convert to physically meaningful units (backscatter coefficients for SAR, brightness temperature for TIR, reflectance for HSI). Apply BRDF corrections and atmospheric compensation for HSI/TIR.
- Temporal matching: create analysis-ready stacks by coalescing acquisitions within a working time window (e.g. ±24 to 72 hours) and flagging temporal offsets that may bias interpretation.
C. Feature engineering
- SAR-derived features: taking the ratio of backscatter intensity measured in VV (co-polarised) and VH (cross-polarised) channels, using fully polarimetric SAR data and decomposing it into physically meaningful scattering components, coherence metrics, InSAR displacement, and soil moisture proxies.
- Thermal-derived features: land surface temperature (LST), thermal inertia, diurnal amplitude, emissivity-normalised metrics, and industrial heat patterns.
- Hyperspectral features: continuum-removed absorption features, spectral signatures (chlorophyll, water, lignin, red edge, clay/carbonate bands), matched filters against spectral libraries, and unmixing coefficients.
D. Fusion layer
- Physics-informed fusion: incorporate radiative transmission models, emissivity models, and dielectric models to constrain inversions.
- Machine learning fusion: use multi-branch neural networks or graph models that ingest modality-specific features, then merge in late or mid-fusion layers, including uncertainty measurements.
- Change analytics: stack multi-temporal fused features to isolate genuine physical change versus acquisition artefacts.
E. Application APIs and products
- Analysis-ready products: soil moisture and crop stress maps, methane/plume detection confidence, mineral prospectivity, infrastructure stability, and illegal deforestation alerts.
- APIs and eventing: publish fused layers and event triggers to enterprise systems (SCADA, EHS dashboards, insurer risk platforms) with SLAs on latency and confidence.
5. High-impact use cases
- Irrigation optimisation: SAR infers near-surface soil soil moisture under clouds, TIR provides evapotranspiration and crop water stress, and HSI reveals nutrient deficiencies and crop type. Together, they enable prescriptive irrigation: watering the right block at the right time.
- Yield forecasting and resilience: fusing thermal stress trajectories with hyperspectral biochemical signals provides early warnings for heat or drought stress, guiding inputs and harvest planning weeks earlier than RGB-only methods.
B. Energy and industry
- Pipeline and facility monitoring: SAR detects ground displacement or terrain instability around assets, TIR flags abnormal heat from flares, leaks, or equipment, and HSI can identify hydrocarbon signatures or chemical residues. Combined, this raises detection sensitivity and specificity while decreasing false alarms.
- Solar and wind performance: thermal maps panel hotspots, SAR assesses foundation and land stability, and HSI identifies soiling and vegetation encroachment. Operators get a consolidated operational view of asset health.
C. Environmental compliance and biodiversity
- Wetland and peatland stewardship: SAR captures hydrologic change, HSI discriminates species and peat condition, and TIR maps microclimate and LST, informing restoration and carbon accounting.
- Deforestation and illegal mining: SAR’s all-weather change detection finds new clearings, HSI distinguishes exposed soils and ore types, and TIR reveals night-time or thermal anomalies caused by operations.
D. Urban infrastructure and risk
- Subsidence and structural monitoring: InSAR quantifies millimetre-scale displacements, TIR helps identify underground leaks via surface-temperature anomalies, and HSI can detect material deterioration. City managers prioritise repairs proactively.
- Disaster response: under smoke or clouds, SAR locates landslides and flood extent, TIR maps active fire fronts, and HSI supports post-event burn severity and contamination assessment.
6. Realistic challenges and how to overcome them
- Co-registration accuracy: misalignment breaks fusion. Commit to robust tie-point matching, sensor models, and DEM-assisted orthorectification. For HSI, account for smile or keystone artefacts.
- Atmospheric and emissivity corrections: TIR and HSI call for careful atmospheric compensation. Emissivity variability can bias LST retrievals; counter this by using emissivity libraries and multi-band fitting.
- Data volume and latency: hyperspectral cubes are heavy; SAR stacks compound the load. Use onboard or edge compression, cloud-native chunked formats (COGs, Zarr), and tiling to keep processing scalable.
- Model generalisation: multi-physics models can overfit to a region or a season. Employ domain adaptation, transfer learning, and continuous calibration using sparse ground truth.
- Uncertainty and explainability: decision-makers need confidence, not just predictions. Produce per-pixel uncertainty layers, data lineage, and human-interpretable features.
7. Build vs buy: a commercial lens
For commercial teams, the question isn’t just, “Can we do this?” but how do we operationalise it profitably?
- Leverage best-of-breed sensors across modalities; few organisations can vertically integrate all three at once. Start where you have a comparative advantage (e.g. high-quality optical payloads) and partner for SAR/TIR access.
- Ship tiered products: a baseline SAR change-detection service, an enhanced SAR+TIR product for operations, and a premium SAR+TIR+HSI diagnostic layer for investigations and planning.
- Focus on latency SLAs and integration (APIs, webhooks) into customer workflows; this is often valued more than marginal accuracy increases.
- Implement data governance and licensing early. Multi-physics stacks cross multiple providers; clarify redistribution rights and derivative product policies.
8. VITO: a toolbox for data fusing
- Operational cloud processing backbone (Terrascope): operated by VITO, Terrascope (a Belgian collaborative ground segment) provides cloud-based access to Copernicus data, avoiding “download-first” workflows. It not only provides access to Copernicus Sentinel data but also to long-term vegetation archives (SPOT-Vegetation, PROBA-V) to support consistent, multi-source time-series analysis.
- Standardised, scalable interfaces for fusion workflows: Terrascope exposes services via standardised APIs, including openEO, that enable scalable “process-close-to-data” fusion pipelines for end users. VITO co-developed and operationalised the openEO platform as a uniform processing interface across multiple European infrastructures.
- Modular, reusable multi-sensor fusion services: VITO invested in a modular, reusable fusion service created for multi-sensor time-series exploitation. They demonstrated Sentinel-1 (SAR) and Sentinel-2 (optical) data fusion, and AI/ML methods to create continuous fused data streams.
- Open-source fusion tooling with production execution paths (FuseTS): a concrete expression of this competence is the FuseTS toolbox: an open-source, Python-based library built to fuse Sentinel time series and run either locally or server-side via openEO. FuseTS exposes “core” fusion and smoothing methods, such as Whittaker smoothing and Multi-Output Gaussian Process Regression (MOGPR).
- Technically mature fusion approach: this shows a technically mature fusion approach: harmonise complementary sensors, reconstruct consistent time series, and then compute downstream indicators.
- Multi-platform fusion: this demonstrates a multi-platform fusion by combining Copernicus Sentinel, drone imagery, and commercial SAR into integrated flood-response products. It uses data harmonisation and probabilistic fusion to manage differences in resolution, uncertainty, and acquisition timing across platforms.
Glancing ahead
As launch costs fall and tasking becomes more dynamic, multi-physics will become the default for serious EO analytics. The winners will not be those with the biggest single sensor, but those who can harmonise physics at scale, compressing diverse signals into timely, trustworthy decisions.
Author: Thys Cronje, Chief Commercial Officer
