Are your data ready for Earth science applications? The new Checklist to Examine AI-readiness for Open Environmental Datasets can help.

As we aim to develop new standards for data readiness for machine learning (ML) and artificial intelligence (AI) applications, there is an opportunity to improve the usability of Earth science data for our own benefit and the benefit of others.

The Data Readiness Cluster is a group including data managers and scientists who use Earth science data on a regular basis to enhance data analysis and develop new applications through AI and ML. These tools can speed up analysis of bigger datasets, offering an understanding of Earth processes across larger regions and timeframes. For example, ML can help assess regional forest biomass, track changes in snow cover across whole mountain ranges and ice sheets, or analyze decadal trends in hard-to-reach field sites where tundra fires affect one of the world’s biggest carbon sinks.

AI and ML are becoming common and more powerful in addressing Earth science research questions. However, the saying “garbage in, garbage out” still applies to even the most high-resolution, well-intentioned big data analyses. Ensuring that the data are AI-ready means that more research questions can be accurately and efficiently addressed, bias is minimized, datasets can be shareable, reusable and discoverable, and the insights gleaned more closely mirror real-world observations.

Since July 2021, the ESIP Data Readiness Cluster has focused on creating tools like the AI-ready checklist for new data managers and other researchers who are learning to prep data for new uses. The checklist continues to evolve alongside the technologies it supports and its current draft covers key questions needed to ensure data preparation, data quality, data documentation and data access. Cluster participants offer their insights along with explanations of the checklist content.

AI-Ready Checklist: Data Preparation

• Have null values/gaps been filled?
• Have outliers been identified?
• Is the data single-source or aggregated from several sources?
• Has the data been gridded (regularized in space and time) or is it in the originally sampled resolution?
• Have targets been identified and labeled (i.e. can this be used as a training dataset for supervised learning techniques)?

AI-Ready Checklist: Data Quality

• Have measures been taken to ensure completeness?
• Are there automated processes to monitor consistency?
• Have measures been taken to reduce bias?
• What is the timeliness of the data? (Near real-time,1 week, 1 month, 1 year, more than 1 year)
  • Is there a difference between raw near real-time access vs fully quality-controlled data that has an additional delay?
• Are there quantitative measures of uncertainty?
• Is there quantitative information about data resolution in space and time?
• Are there published data quality procedures or reports? (Link to reports)
• Is the provenance tracked and documented?
• Are there checksums / other checks for data integrity?
• How big is the dataset? Depending on the resource, this might be total data volume, dimensionality, number of images, data files, table rows, image size, etc.
• Is this essentially raw data or a derived/processed data product?
• Is this observational data or simulation/model output?
• Has the data been peer-reviewed?
• Has it been down-sampled to reduce resolution, or is it raw? If so, are the raw data available?

Data documentation is about context, use and interoperability.

From finding the data, to knowing how to use them, to respecting the bounds of licenses and privacy, the final sections of the Data Readiness Cluster’s AI-ready checklist offer up questions about how to find and use a dataset.

Documentation is not synonymous with standardization, but the two go hand-in-hand. Returning to our dishes metaphor, well-documented data knows its ice cream bowls from its teacups and that the traditional dinner plate is 10.5 inches across. Data documentation would love cupboard labels and a table of contents, too. While it may sound silly in a household context, there are many steps and categorizations that researchers take for granted in data generation, much like how we all get used to knowing where the pizza cutter goes in our own kitchen. The checklist’s documentation questions offer a map and point to the right people to ask when putting away the data.

True color satellite imagery gives a sense of the vastness of Earth’s features. To render such images, data has to be carefully documented and analyzed. Credit: NOAA/NASA, Suomi NPP 2016

Data access is the key to being FAIR.

Findable, accessible, interoperable and reusable. FAIR is the open science standard for shareable data.

The principles are not all encompassing – in fact FAIR can be at odds with other frameworks like CARE that emphasize community sovereignty and use. But if AI-ready data as a whole is a dishwasher full of clean dishes ready to get put away, then FAIR helps make sure there is not a lock on the silverware drawer.

Likewise, access can have layers: Knowing the right cupboard is one step, and not having a jumble inside makes it easier to set the table more tidily and efficiently. With this in mind, broader data access invites more users to dinner. Knowing when and where to show up helps diners and data alike, even if some of those users never step foot in the kitchen.

As the final steps of the checklist, data access gets at the whole process’ main goal: AI-ready Earth science data strives to be information that people can use.

This blog post was co-written by the Data Readiness Cluster participants, Allison Mills from ESIP and Jennifer Fulford from NOAA.

ESIP stands for Earth Science Information Partners and is a community of partner organizations and volunteers. We work together to meet environmental data challenges and look for opportunities to expand, improve, and innovate across Earth science disciplines.

Learn more esipfed.org/get-involved and sign up for the weekly ESIP Update for #EarthScienceData events, funding, webinars and ESIP announcements.