Disclaimer & Assessment Methodology
This score was produced via LLM-assisted assessment of the repository structure and the supplied evidence bundle. These findings should be treated as a directional pilot signal rather than a verified rating. For more details on our pilot objectives and the development path toward rules-based human review, see the Development Roadmap. Raw files are accessible at the assessment report source.
Global Forest Watch Assessment
"Tree-cover loss and related forest indicators can be monitored globally."
The wri/gfw repository provides strong open-source evidence for the Global Forest Watch web application itself: it is public, MIT-licensed, documented for local setup, and includes calculation notes, API/layer schema documentation, CI, CodeQL, and release/deployment practices. Traceability is much weaker for the underlying environmental evidence chain: the bundle points to external Resource Watch API datasets and gives some formula-to-column links, but it does not include primary measurement provenance, dataset licenses, uncertainty/quality-control documentation, versioned data snapshots, exact production workflow runs, or explicit claim-to-data-to-code-to-review linkages. Overall, this is a substantially open web-app and methods repository with only partial end-to-end open traceability for the environmental claims displayed by the system.
Claim Traceability Pathway
Inspection trail mapping the connections from raw inputs to the published claim.
Traceability Health Summary
Scan diagnosis of where the evidence chain stands and what requires attention first.
Open-Source Models, Methods, and Software
This stage provides solid evidence, making calculations and documentation highly inspectable and reusable for external actors.
Open Linkage
This is the weakest segment of the evidence chain. Gaps in records or lack of linkage break the evidence chain here.
- Unique dataset DOIs & licenses
- Top-level LICENSE files
- Execution run configs & workflow logs
- Scrutiny logs & replication results
- Open access publications
- Dynamic provenance graph mapping
Strengthen Open Linkage
Create a versioned provenance graph mapping every step of the computation from source data to claim.
Evidence Linkage Summary
Open Linkage is not a standalone category; it is the connectivity of the evidence chain. Below is the connection status of the connection links between each step in this project's journey.
Are data input paths and schemas programmatically defined inside the code files or configurations?
Is code execution managed through versioned workflow scripts (e.g. Snakemake pipelines) rather than manual scripts?
Are execution outputs and logs tested automatically (CI/CD) and verified inside public pull request reviews?
Are peer reviews, community issues, and discussions linked directly to published methodology reports?
Does the final published paper cite the exact dataset identifiers, model hashes, and run logs that produced the results?
There is some linkage across the evidence chain: the README links the web app to the Resource Watch API endpoint for GFW datasets, API documentation explains how metadata keys and layer configurations connect API records to application behavior, and calculation docs connect certain widget outputs to formulas and, in one case, to a named source column and table identifier. This prevents a very low score. However, the chain is not explicit, specific, versioned, or externally verifiable end-to-end. The supplied evidence does not link particular environmental claims or dashboard values to exact dataset versions, primary measurement records, code commits, model configurations, workflow runs, CI artifacts, review discussions, publication versions, or archived outputs. Open components exist, but the exact evidence chain behind a given claim is not inspectable from the bundle.
Detailed Dimension Inspection
Granular findings for each processing step in the Open Traceability framework.
Open Input Data and Measurement Evidence
Evidence Found
The evidence shows that GFW input layers and datasets are at least identifiable through the Resource Watch API, with an endpoint for GFW-specific datasets and documentation of dataset, layer, vocabulary, and metadata fields. Some derived-widget documentation names specific source columns and a geotrellis table identifier for aboveground biomass, and widget documentation describes use of raw weekly aggregated alert counts. This prevents a lower score because the repository does not treat data sources as entirely opaque.
Gaps Identified
However, the evidence bundle does not include the actual primary datasets, dataset licenses, measurement protocols, quantified errors/uncertainties, quality-control records, data versioning, or traceability from secondary data back to primary satellite or field observations. The score is therefore in the partial-to-moderate range rather than high.
"“Map layers and relevant datasets are stored in the RW-API” and a GFW dataset endpoint is provided: https://api.resourcewatch.org/v1/dataset?app=gfw&includes=layer,vocabulary,metadata&page[size]=200."
Relevance: Shows that input datasets and layers are at least externally identifiable through a public API endpoint."Documents dataset metadata fields such as `info.name`, `info.description`, `info.metadata`, layer `iso`, `applicationConfig`, and `legendConfig`."
Relevance: Supports some documentation of how datasets and layers are described and surfaced in the application."Aboveground biomass is said to come from the `whrc_aboveground_biomass_stock_2000__Mg` column within summary geotrellis tables, with example table `998dd97a-389f-4a02-988f-17b184f507ac`."
Relevance: Provides a concrete trace from a derived carbon calculation to a named data column and table identifier."Describes calculations using “raw, weekly aggregated data” and alert counts `alert__count` for GLAD and Fires widgets."
Relevance: Indicates the form of some input data used in dashboard calculations, though not the primary measurement provenance.Open-Source Models, Methods, and Software
Evidence Found
The web application source is public on GitHub, has an MIT license, and the README provides the main technology stack and setup commands. The repository includes method documentation for climate and widget calculations, API/layer schema documentation, component examples, and CI/security workflow files. This is strong evidence that a substantial part of the analytical and presentation logic can be inspected and reused. The score is not higher because the evidence bundle does not include the full dependency manifest, lockfile, backend/API source, data-processing pipelines, geotrellis generation code, infrastructure-as-code, or complete model implementations behind the environmental datasets. The README also states that the app depends on external APIs and Heroku deployments, so the software infrastructure evidenced here is not fully independently operable from the repository alone.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Repository description: “Global Forest Watch: An online, global, near-real time forest monitoring tool”; license: MIT; topics include deforestation, forest-monitoring, satellite-imagery, Next.js, React, Redux."
Relevance: Establishes public source repository, project purpose, and open-source license metadata."The repository uses the MIT License, granting permission to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software."
Relevance: Supports open-source reuse of the software under a recognized permissive license."States the app is built with Next.js, React, and Redux, and provides commands to clone, install dependencies with `yarn`, copy `.env.sample`, and run `yarn dev`."
Relevance: Shows that the software stack and basic run method are documented."Documents formulas such as belowground biomass `M_bgb = 0.489 x M_agb^0.89` and carbon stock `M_Carbon = 0.5 x M_biomass`."
Relevance: Provides inspectable analytical methods for some derived environmental quantities."Documents statistical bands, weekly aggregation, zero-filling, mean, standard deviation, and smoothing for fires and GLAD widgets."
Relevance: Shows that some dashboard analytical logic is documented outside code.Open Execution and Reproducibility
Evidence Found
The evidence supports basic software execution reproducibility: the README gives installation and development-server instructions, identifies deployment branches and environments, and GitHub Actions specify a Node version with lint and unit-test commands. CodeQL adds a scheduled/publicly defined analysis workflow. This prevents a low score because an external actor can plausibly run and test the web application. The score is not higher because the evidence does not provide a complete computational workflow from environmental input data to reported outputs, exact production run records, pinned API/data versions, containerized environments, environment-variable values, archived outputs, or re-executable data-processing pipelines. The environmental calculations depend on external APIs and datasets whose runtime state is not captured in the supplied evidence.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Provides commands: clone the repository, run `yarn`, copy `.env.sample` to `.env.local`, run `yarn dev`, and access the app at `http://0.0.0.0:3000`."
Relevance: Shows basic steps for an external actor to execute the web application locally."Describes Gitflow: pull requests merge into `develop`, staging/pre-production deploy automatically, and merging `develop` into `master` triggers production deployment."
Relevance: Documents the deployment path from code branches to running environments."GitHub Actions runs on pull requests and pushes to `master`/`develop`, uses Node `18.15.0`, installs modules with `yarn`, runs ESLint, and runs `yarn test:ci`."
Relevance: Provides public, inspectable automated execution for tests and linting with a specified Node version."CodeQL analysis runs on pushes, pull requests, and a weekly schedule for JavaScript."
Relevance: Shows a repeatable automated security/static-analysis workflow, though not environmental-result reproduction.Open Community and Review
Evidence Found
There is evidence of public development infrastructure: the GitHub repository has open issues, pull requests trigger CI, review apps are automatically deployed for pull requests, releases are managed through GitHub releases, BrowserStack is used for cross-browser testing, and CodeQL is configured. This shows some visible channels for scrutiny of software changes. The score is not higher because the evidence bundle does not include actual issue discussions, pull-request reviews, governance records, peer-review records, methodological debates, correction notices, replication attempts, community forum records, or evidence of affected-community feedback. The review evidence is therefore mostly procedural rather than a demonstrated public record of scrutiny around environmental claims.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Repository metadata reports `open_issues_count`: 16."
Relevance: Indicates the existence of a public issue tracker, although issue content is not supplied."States that pull requests merge into `develop`, and Heroku Review Apps deploy automatically when a pull request is created with a link added to the respective pull request."
Relevance: Shows a visible software-review workflow for proposed changes."CI runs lint and unit tests on pull requests to `develop` and `master`."
Relevance: Supports automated review/checking of contributions before merging."States that GitHub releases are used to record changes and that semantic release logs group commits into major, minor, and patch sections."
Relevance: Shows a public mechanism for communicating software changes, though actual release records are not included."“We use BrowserStack to find and fix cross-browser issues.”"
Relevance: Shows an additional testing/review practice for the web application.Open Publications and Communication
Evidence Found
The project has a public homepage, a public GitHub README explaining that GFW is a dynamic online forest monitoring and alert system, and documentation for API layer/dataset configuration and widget calculations. These materials support public communication and some scrutiny of displayed indicators and methods. The score is not lower because the documents include concrete formulas and schema explanations rather than only marketing descriptions. The score is not higher because the evidence bundle does not include specific reports, papers, dashboard pages, indicator version histories, update-cycle documentation, data-source citations in final outputs, review status of publications, or open content licensing for the website/publications. For a dynamic dashboard, the supplied evidence is insufficient to verify that all claims are clearly tied to sources, update cycles, and versions.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Repository homepage is listed as `https://www.globalforestwatch.org`."
Relevance: Shows a public user-facing publication/dashboard endpoint for the project."Defines GFW as “a dynamic online forest monitoring and alert system that empowers people everywhere to better manage forests.”"
Relevance: Provides a public communication statement about the environmental monitoring purpose of the app."Documents how GFW layers and datasets are represented in the API, including metadata, legends, interactions, and external links in `applicationConfig.moreInfo`."
Relevance: Supports scrutiny of how dashboard layers are defined and communicated to users."Explains carbon-stock and biomass calculations used in the `carbonStock` widget with equations and notes on rounding/biological correctness."
Relevance: Provides explanatory material for at least one class of published dashboard output."Outlines mathematical and statistical calculations in GFW dashboard widgets, including fires and GLAD statistical bands."
Relevance: Supports public understanding of methods used in communicated dashboard indicators.Open Linkage
Evidence Found
There is some linkage across the evidence chain: the README links the web app to the Resource Watch API endpoint for GFW datasets, API documentation explains how metadata keys and layer configurations connect API records to application behavior, and calculation docs connect certain widget outputs to formulas and, in one case, to a named source column and table identifier. This prevents a very low score.
Gaps Identified
However, the chain is not explicit, specific, versioned, or externally verifiable end-to-end. The supplied evidence does not link particular environmental claims or dashboard values to exact dataset versions, primary measurement records, code commits, model configurations, workflow runs, CI artifacts, review discussions, publication versions, or archived outputs. Open components exist, but the exact evidence chain behind a given claim is not inspectable from the bundle.
"States that map layers and relevant datasets are stored in RW-API and provides a GFW-specific API endpoint including datasets, layers, vocabularies, and metadata."
Relevance: Shows a partial linkage between the frontend application and the dataset/layer source system."Explains that dataset metadata contains an `info.metadata` key that “populates metadata modal” and layer configuration controls legends, interactions, and links."
Relevance: Shows how API metadata is intended to connect datasets to user-facing dashboard presentation."Links the `carbonStock` widget to aboveground biomass in column `whrc_aboveground_biomass_stock_2000__Mg` and documents formulas for belowground biomass and carbon stock."
Relevance: Provides a concrete, though incomplete, link between an input field and a derived widget output."States that GitHub releases record changes to the app and tag commits as major, minor, patch, or ignore."
Relevance: Suggests some versioning mechanism for software changes, but not linked to exact environmental claims or data versions."CI runs lint and unit tests on pushes and pull requests for `master` and `develop`."
Relevance: Shows inspectable execution checks, but no linkage from CI runs to specific published environmental outputs.Critical Traceability Gaps
Inspection details of where the evidence chain breaks down and what that implies for public trust.
| Identified Gap | Impact on Traceability | Suggested Remediation |
|---|---|---|
| Dataset not linked or versioned | High ‐ Blocks inspection of inputs | Register raw datasets in Zenodo and cite unique DOIs in config files. |
| Missing root license or dependencies | Medium ‐ Restricts model inspectability | Add a LICENSE file (e.g. AGPL-3.0) and export environment lockfiles. |
| Workflow run details not archived | Critical ‐ Restricts computational reproducibility | Archive Snakemake workflow runs and publish log files on Zenodo. |
| Review and replication logs hidden | Medium ‐ Hides validation gate process | Maintain an open-source technical review tracker in public channels. |
| Publication lacks direct evidence citation | High ‐ Claims are unlinked to runs | Add structured CITATION.cff and link claim statements to run hashes. |
Remediation & Improvements Roadmap
Actionable roadmap prioritized by implementation complexity.
Quick Wins
Low effort, high immediate return.
- Add CITATION.cff file
Enables clear publication-to-repository citation schema.
- Commit top-level LICENSE
Resolves legal terms of software and dataset inspectability.
- Link communication channels
Add status/discord badges to repository README files.
Medium Effort
Standard engineering practices.
- Specify configurations in config files
Provide explicit parameters and dataset configurations in repository.
- Lock computational environments
Lock dependencies using Conda-lock or Docker image digests.
- Archive automated workflow logs
Publish CI test summaries and solver objective check values.
Strategic Upgrades
Highest traceability return.
- Publish workflow runs on Zenodo
Create persistent, immutable archives of the full execution outputs.
- Build claims-to-evidence provenance graphs
Establish a machine-readable map of the end-to-end evidence trail.
Framework Dimension Definitions
- Open Input Data and Measurement Evidence: Identifiable, documented, attributable, versioned raw measurement evidence inputs.
- Open-Source Models, Methods, and Software: Inspectability of Analytical logic via code repositories, configurations, and license files.
- Open Execution and Reproducibility: Computational environments, workflows, run logs, and outputs make rerun processes inspectable.
- Open Community and Review: Scrutiny, peer review, issue tracking, and responses to replication challenges are visible.
- Open Publications and Communication: Resulting reports, papers, and dashboards are openly accessible and reference source materials.
- Open Linkage: The versioned, explicit connectivity linking data, models, execution, review, and publication steps.
Human Review Notes
Human review of this assessment report is pending. An independent reviewer must inspect the repository references to verify the findings.