What are the major challenges of using earth observation (EO) data for impact evaluation?

Earth observation involves collecting information about the chemical, physical, and biological systems of the earth through remote sensing technologies. This information is gathered with the help of satellites carrying imaging devices that are used to monitor, measure, and understand the status of the manmade and natural environment.

In Australia, the majority of earth observation data is operated by the researchers and government. Disaster management, weather forecasting, bushfire management, and environmental management are examples of public sector usages of earth observation data. Also, the usage of developed technologies for remote sensing like electro-optical sensors and Synthetic Aperture Radar (SAR) among others is driving the growth of the market. 

As well as the growing government expenditure is also boosting the growth of the market. In addition to this, according to the research report of Astute Analytica, the Australia Earth Observation (EO) Market is growing at a compound annual growth rate (CAGR) of 3.5% during the forecast period from 2022 to 2030.

The Major Challenges of Using Earth Observation (EO) Data for Impact Evaluation are: -

Reference data

Satellite data can be utilized to indicate measures of technology adoption or outcomes in areas that are difficult or costly to access with ground-based data collection techniques. They can also be utilized to back-cast such measures to other times covered in the satellite record. Also, for satellite-based measures to be valid, it is necessary to carefully prepare and validate remote sensing models. The collection of high-quality reference data is often costly and may have substantial implications for project budgets.

Measurement errors

Errors in model outputs or remote sensing products, developed from them, can introduce prejudice into the econometric analyses that are part of impact evaluation. 

For instance, in one of the examined case studies, researchers aimed to understand how the adoption of zero-tillage practices influenced the production of smoke from agricultural fires in India. To quantify effects, the research team intended to utilize remote sensing data to measure both smoke outcomes and zero-till adoption. Also, they quickly learned that smoke from fires covered satellite images of the land surface. 

As a result, satellite-based calculations of zero-till adoption were less precise in areas concerned by smoke. Because errors in their adoption measure were correlated with their primary outcome of interest. This situation is known as a non-classical measurement error.

Understanding the data-generating process

In evaluating the possibility for remote sensing data to be utilized in an impact evaluation, researchers are required to think carefully about the precise pathways through which technology adoption is hypothesized to impact outcomes. In econometric terms, these pathways are directed to as data-generating processes. The nature of the data-generating process has many significances for research design. 

For instance, if a given technology delivers impacts of very small spatial scales, then it may only be applicable to operate high-resolution satellite data. Alternatively, if there is a pause between when a technology is adopted and when impacts happen, then evaluating impacts could need remote-sensing products with satisfactory temporal coverage.