Integrity in Data: How Earth Science Data Groups Are Shaping Practical AI-Use Policies

This blog is a collaborative creation from the contributors to the Machine Learning Cluster and others in the ESIP community with ideas to share about approaching the building AI/SI use policies in an era of change. See footnote for more on the contributors.

Across the Earth science data community, researchers are bumping into the same dilemma. One recent discussion in the ESIP Slack workspace went something like this: “Our group needs an AI use policy to outline when and how we should use tools like Claude or GitHub Copilot. We are a small academic group managing environmental data, not a massive agency like NASA or an AI research center. How do we establish sensible boundaries without getting buried in bureaucracy?”

Our community is at a defining moment for modern scientific data governance. Major federal agencies are deploying top-down institutional frameworks, but smaller academic groups, data repositories, non-profits, and domain-specific research labs are finding themselves in uncharted territory while using Artificial Intelligence (AI; also called Super-Intelligence or SI) to write Python scripts, manage and format metadata, summarize literature, or streamline workflows.

Adopting AI/SI tools raises pressing questions. How do we ensure data integrity when language models still hallucinate? How do we protect sensitive environmental or proprietary data from leaking into public training sets? And how do we prevent early-career researchers from losing foundational technical skills by over-relying on automated tools?

Our community is at the center of a thoughtful, grassroots movement to answer these questions. Rather than waiting for a single, universal mandate, Earth science data communities like ESIP are curating a mosaic of community-led frameworks and guidelines to craft practical, values-aligned AI/SI policies.

Why one size doesn’t fit all

One of the primary friction points in developing AI/SI use guidelines is that not every group shares the same operational realities, recently discussed in a white paper authored by astronomy and data researchers. The paper demonstrates that many research groups have fundamentally different priorities, which makes standard off-the-shelf policies inadequate. This analysis outlines several research laboratory archetypes, each balancing competing values:

• High leverage and productivity: Labs that prioritize maximizing publication output, rapid script iteration, and workflow automation.
• Craftsmanship and skill development: Groups centered on training students and postdocs where learning how to write code, curate metadata, and inspect raw data from scratch is as important as the end result.
• Trustworthiness and rigor: Environments where verified accuracy, reproducible scientific workflows, and mathematical rigor take precedence over speed.
• Data stewardship: Archives and data repositories focused on long-term data preservation, metadata standards, provenance tracking, and strict data governance.

In Earth science, groups are realizing that AI/SI policies aren’t one-size fits-all, and the archetypes above aren’t necessarily static or mutually exclusive. For example, one member of a small academic research group might use an automated AI/SI assistant to generate a quick script to reformat ocean salinity measurements in seconds. However, if another lab member relied entirely on AI/SI to manage the same measurements without understanding the preceding transformations, subtle errors in data provenance or unit conversions could pass unnoticed into public archives. AI/SI policy generation cannot be a passive exercise, but instead it  requires aligning AI/SI adoption with the specific mission and values of the organization in question.

From Overload to Action

When research leaders begin drafting an AI/SI policy, the sheer volume of choices often leads to policy paralysis. Should ChatGPT be banned from the manuscript writing process? Should GitHub Copilot be mandatory for data cleaning?

Next, organizations can focus on asking “What problems are we trying to solve?” and identifying use cases, which the authors of “A Path Through AI Overwhelm” identify as a successful approach. Earth science groups could consider using the following functional buckets when organizing their policies around use-case categories:

1. Administrative and efficiency tools: Summarizing lengthy committee meetings, polishing grant proposals, or drafting routine email communications. (Generally perceived as low risk provided no confidential data is exposed.)
2. Data wrangling and code generation: Drafting Python, R, or SQL scripts to clean, reformat, and ingest environmental data. (Perceived as medium risk; may require mandatory human code review and local testing.)
3. Synthesis and scientific writing: Using language models to assist in drafting literature reviews, manuscript outlines, or documentation. (Perceived as needing high scrutiny; requires strict disclosure and verification of references.)
4. Mission-critical analytics and metadata creation: Generating public-facing dataset documentation, catalog tags, or automated quality control flags. (Perceived as high risk; requires explicit human verification to protect data provenance.)

Organizations like Intertidal and non-profit data consortia have adopted similar tiered approaches. By grouping tasks into clear “do's, don'ts, and caution zones,” much like Candid’s high-level guidance—groups can grant team members autonomy while establishing firm guardrails around high-stakes data assets. However, simplistic judgments about which AI/SI uses are more or less risky may not be possible and are needed as a point of discussion for each unique AI/SI use policy. Many use cases could have short- and long-term risks, or low-and high-probability risks. Instead, evaluations of AI/SI risk may be more accurately thought of as tradeoff analysis of different types and timescales of risk.

Open science, ethics, and the integrity horizon

In Earth science data – which underpin global climate models, policy, and public safety – scientific integrity and open science principles are non-negotiable. We can draw insights on this from broader academic movements, such as the Leiden Declaration on Artificial Intelligence and Mathematics. This declaration was drafted by an international working group of university researchers, created to establish community norms and safeguard foundational values of mathematical research as AI/SI tools become increasingly integrated into the field. 

The Leiden Declaration emphasizes that while AI/SI can accelerate research, human researchers must retain full, unalienable responsibility for the accuracy and integrity of their work. Key points being integrated into Earth science discussions include:

• Human accountability: AI/SI tools cannot be named as authors or creators of scientific outputs or dataset documentation. Human stewards remain solely accountable for errors, hallucinations, or misinterpretations.
• Transparent disclosure: Researchers should explicitly document when, where, and how AI/SI tools were used in generating software, workflows, or text.
• Rigorous attribution: Generative models frequently rephrase published work without attribution. Scientific data groups must exercise proactive effort to ensure prior data creators, domain specialists, and literature sources receive appropriate credit.
• Equity and access: Over-reliance on expensive, proprietary commercial models risks widening the gap between well-funded institutions and smaller community repositories. Prioritizing open-source, community-adapted models aligns with the foundational ethos of open science.

This commitment to transparency is also reflected in institutional projects such as the draft University of Illinois Urbana-Champaign’s AI Guidance Initiative, championed by members of the Faculty Senate IT Committee. By bringing faculty, computational experts, and data managers into a shared conversation, university-wide and group-level policies are evolving to ensure that software and data management workflows remain FAIR (Findable, Accessible, Interoperable, and Reusable).

Building a living policy across changing data landscapes

Perhaps the most encouraging trend in the Earth science data community is the recognition that while organizational principles guiding AI/SI use should be consistent, an AI/SI use policy is not a static document to be written once and never revisited. As tools evolve and model capabilities change, policies must function as living frameworks to ensure groups are adhering to their principles.

Communities like ESIP’s are currently synthesizing diverse threads and combining white papers, non-profit templates, academic declarations, and real-world workflows into adaptable community drafts. The goal is not to impose a top-down mandate, but to empower every group with a practical toolkit. Thanks to thought leadership in this area across many disciplines, a draft ESIP AI/SI Use Policy is in the works to ensure we follow FAIR and CARE principles, and that solid data security and privacy guidelines are an essential baseline. ESIP community members and partners may also need to consider how and when community work should follow TRUST principles.

Join the conversation

For any Earth science data group thinking about developing their AI/SI Use Policy, the path forward starts with open dialogue. Gather your team, define your core scientific values and principles, ask where AI/SI can genuinely add value without unacceptable tradeoffs, and build something that fits your unique mission. In the rapidly shifting landscape of artificial intelligence, thoughtful stewardship remains our most valuable asset. 

Join the ESIP Community on Slack (particularly the #machine_learning_ai channel) or in one of our Collaboration Areas currently discussing machine learning and AI/SI or data stewardship. Want to convene the minds talking about this topic? Submit a session proposal for the ESIP January Meeting. Only by working through this changing landscape together will Earth science researchers make thoughtful decisions around how AI/SI plays a part in our community.

——-

Drafted with the assistance of AI: Google Gemini was fed Slack chats and links, told to cite the sources, extract main takeaways from resources, and create an outline for this blog. The blog was then edited for accuracy, content, and tone by humans. Contributions, ideas, and edits from talks with Kelly Lockhart, Kate Wing, Karen Stocks, Althea Marks, Daniel S. Katz, Sarah Cooley, and contributors from the ESIP Machine Learning Cluster.

ESIP (Earth Science Information Partners) is a community of partner organizations and volunteers. We work together to meet environmental data challenges and find opportunities to expand, improve, and innovate across Earth science disciplines. Learn more and sign up for the weekly ESIP Update for #EarthScienceData events, funding, webinars and ESIP announcements.