Disclaimer & Assessment Methodology
This score was produced via LLM-assisted assessment of the repository structure and the supplied evidence bundle. These findings should be treated as a directional pilot signal rather than a verified rating. For more details on our pilot objectives and the development path toward rules-based human review, see the Development Roadmap. Raw files are accessible at the assessment report source.
PyPSA-Eur Assessment
"An open sector-coupled optimisation model of the European energy system."
PyPSA-Eur shows strong Open Traceability overall for an open European energy-system optimisation model: its public repository, open-source licensing statements, dependency/environment files, Snakemake-based workflow description, CI/integration tests, validation workflows, community contribution processes, documentation, citation metadata, Zenodo links, and explicit limitations make much of the evidence and method chain externally inspectable. The strongest dimensions are open-source software and public communication, while input data and reproducibility are also well supported by named data sources, versioning infrastructure, and automated workflows. The main traceability gap is exact end-to-end linkage for individual environmental claims or published results: the supplied evidence does not include a complete provenance record tying a specific claim to exact primary data versions, code commit, model configuration, solver settings, workflow logs, output files, review records, and final publication artifact.
Claim Traceability Pathway
Inspection trail mapping the connections from raw inputs to the published claim.
Traceability Health Summary
Scan diagnosis of where the evidence chain stands and what requires attention first.
Open-Source Models, Methods, and Software
This stage provides solid evidence, making calculations and documentation highly inspectable and reusable for external actors.
Open Linkage
This is the weakest segment of the evidence chain. Gaps in records or lack of linkage break the evidence chain here.
- Unique dataset DOIs & licenses
- Dynamic provenance graph mapping
Strengthen Open Linkage
Create a versioned provenance graph mapping every step of the computation from source data to claim.
Evidence Linkage Summary
Open Linkage is not a standalone category; it is the connectivity of the evidence chain. Below is the connection status of the connection links between each step in this project's journey.
Are data input paths and schemas programmatically defined inside the code files or configurations?
Is code execution managed through versioned workflow scripts (e.g. Snakemake pipelines) rather than manual scripts?
Are execution outputs and logs tested automatically (CI/CD) and verified inside public pull request reviews?
Are peer reviews, community issues, and discussions linked directly to published methodology reports?
Does the final published paper cite the exact dataset identifiers, model hashes, and run logs that produced the results?
The project has moderate-to-strong linkage evidence across the chain: the README connects the model claim to named input datasets, Snakemake workflow construction, documentation, limitations, Zenodo records, releases, and papers; CITATION.cff identifies a specific version and repository; the data-versioning validation layer implies structured links among datasets, versions, sources, tags, dates, and URLs; and CI workflows connect code revisions to tests, validation, release preparation, and container images tagged by commit SHA. However, the evidence does not show an explicit, externally inspectable provenance graph from a specific environmental claim or publication result to exact data versions, code commit, model configuration, solver environment, workflow run logs, outputs, review records, and publication artifacts. The linkage is therefore substantially present at project-infrastructure level but incomplete at exact claim/run level. It is not lower because several artifacts deliberately support versioning and cross-referencing; it is not higher because exact end-to-end claim-level traceability is not demonstrated in the supplied bundle.
Detailed Dimension Inspection
Granular findings for each processing step in the Open Traceability framework.
Open Input Data and Measurement Evidence
Evidence Found
PyPSA-Eur provides strong public evidence that major model inputs are identifiable and based on open or externally accessible sources: the README lists the power grid model, power plant database, ENTSO-E demand, ERA5/SARAH-3 renewable time series via atlite, land eligibility analysis, and Eurostat/JRC-IDEES energy balances. The repository also includes dataset documentation for at least one input layer, entsoegridkit, with fields, provenance, processing caveats, and warnings about inaccuracies. There is evidence of a data-versioning layer through tests validating data/versions.csv, including source categories, archive URLs, tags such as latest/supported/deprecated, and consistency checks. The score is not higher because the evidence bundle does not include the actual data/versions.csv, complete source manifest, licences and access conditions for every input, quantified uncertainty/error estimates for all primary datasets, or full transformation documentation from primary to model-ready data. It is not lower because the supplied evidence clearly shows a systematic, documented open-data approach and explicit caveats for input quality.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"States that the model is built from open data using a Snakemake workflow and lists major datasets including OpenStreetMap, powerplantmatching, ENTSO-E Transparency Platform, ERA5/SARAH-3 via atlite, and Eurostat/JRC-IDEES."
Relevance: Shows identifiable public input sources and the project’s open-data basis."Documents an unofficial ENTSO-E-derived GridKit dataset, its March 2022 map extract origin, CSV contents and fields, and cautions about coordinate inaccuracies, voltage ranges, derived transformers, and nearest-station generator connections."
Relevance: Provides concrete provenance, schema, and quality caveats for an input dataset."Defines validation for data/versions.csv with dataset, version, source, tags, added date, note, URL, required archive URLs, and consistency checks for latest versions."
Relevance: Shows that the project maintains and tests a structured data-versioning layer, although the actual CSV is not included in the evidence bundle.Open-Source Models, Methods, and Software
Evidence Found
The project has excellent public software traceability: it is hosted in a public GitHub repository, describes itself as fully open source, provides citation metadata identifying version v2026.02.0 and MIT licensing, includes SPDX licence headers in supplied source/test files, lists a full Conda environment with modelling and geospatial dependencies, and provides a Dockerfile for a development environment. The repository uses version control, release badges, dependency-management tooling, and open-source frameworks such as PyPSA, Snakemake, atlite, and powerplantmatching. The score is not higher because GitHub repository metadata reports no repository-level licence, the evidence bundle does not include the root LICENSE file or pixi.lock, and environment.yaml uses version ranges rather than a fully locked environment. It is not lower because the available evidence is explicit, reusable, and strongly supports inspection and operation of the model software.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Public GitHub repository PyPSA/pypsa-eur with description “PyPSA-Eur: A Sector-Coupled Open Optimisation Model of the European Energy System,” default branch master, and active issue count."
Relevance: Establishes public version-controlled availability of the model/software."Identifies the project title, repository URL, version v2026.02.0, authors, and licence MIT."
Relevance: Provides citation, version, repository, authorship, and open-source licence evidence."Lists Conda channels and dependencies including pypsa, snakemake-minimal, atlite, linopy, geopandas, rasterio, entsoe-py, powerplantmatching, solvers, and many geospatial/scientific packages."
Relevance: Documents the software dependencies needed to understand and run the model."Builds a development environment from a Pixi base image, copies pixi.toml and pixi.lock, installs the default environment, and labels the source repository."
Relevance: Shows reusable infrastructure software for operating the development environment.Open Execution and Reproducibility
Evidence Found
The project has strong execution-traceability infrastructure: the README says the model is built from open data using a Snakemake workflow; the environment file documents dependencies; CI runs unit tests, integration tests, pylint checks, and Snakemake test workflows; validation workflows run self-hosted model validation and generate result comments; release workflows update documentation DAGs; and container-image workflows tag development environments by commit SHA. These artifacts make the computational pathway substantially inspectable and partly re-executable. The score is not higher because the evidence bundle does not include the Snakefile/rules, exact scenario configuration for a published environmental claim, full logs, result artifacts, solver settings, random seeds if any, or a specific run record linking inputs to outputs. It is not lower because there is clear evidence of automated tests, integration workflows, environment specification, and reproducibility-oriented workflow tooling.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"States that the model is built from open data using a Snakemake workflow and is fully open source."
Relevance: Identifies the workflow system connecting inputs to model construction."Runs unit tests, integration tests on multiple operating systems, source-change filtering, pylint checks, Snakemake test workflows, and uploads logs/results artifacts with three-day retention."
Relevance: Shows automated execution checks and partial preservation of run artifacts."Runs pypsa-validator on pull requests with a specified environment file and Snakemake config, then creates a report comment with plots such as energy, costs, and balances."
Relevance: Shows model validation execution and reporting around proposed changes."Builds and pushes a development environment image to ghcr.io/pypsa/eur-dev-env tagged with the GitHub commit SHA, or retags an existing image for the current SHA."
Relevance: Supports re-execution by tying container environments to code revisions.Open Community and Review
Evidence Found
The project provides strong evidence of open community and review mechanisms: contri
Gaps Identified
buting documentation welcomes bug reports, ideas, issues, pull requests, help-wanted work, and Discord discussion; it explicitly says all code contri
"Welcomes ideas, bug reports, code contributions via pull requests, use of help-wanted issues, draft PRs, and Discord; states that all code contributions follow the four-eyes principle and are reviewed by a second person before incorporation."
Relevance: Directly documents public community channels and review policy."Repository metadata reports 256 open issues."
Relevance: Indicates an active public issue tracker for critique, bug reports, and discussion."Runs CodeQL analysis on pushes, pull requests, and a weekly schedule for Python."
Relevance: Shows automated code review/security scrutiny as part of the public development process."Generates an SBOM on master, performs vulnerability scans, uploads SARIF, and checks fork PRs for new high-severity CVEs against a baseline."
Relevance: Shows visible automated dependency/security review processes.Open Publications and Communication
Evidence Found
PyPSA-Eur has strong publication and communication traceability. The README is openly accessible, clearly states the model’s purpose and scope, lists the evidence base and major data sources, includes warnings and limitations, links to documentation, release badges, Zenodo DOIs, and example peer-reviewed or scholarly outputs. CITATION.cff provides a citable project record with version, authors, repository, and licence. Documentation files in the bundle use CC-BY-4.0 SPDX headers, supporting reuse of explanatory material. The score is not higher because the bundle does not include the full documentation, complete dashboard/update-cycle documentation if any, full release notes, paper licences, or a complete mapping from each publication claim to exact artifacts. It is not lower because the main public communication artifacts are open, well labelled, citable, and explicitly point users to limitations and supporting resources.
Gaps Identified
No significant gaps identified. This stage is fully documented and verifiable.
"Provides model overview, source list, diagrams, warnings, documentation link, release badge, Zenodo DOI badges, and links to papers and presentations showing model capabilities."
Relevance: Shows clear public communication of claims, scope, evidence base, documentation, and limitations."States how to cite the project, gives title, repository, version v2026.02.0, licence MIT, authors, and ORCIDs for many authors."
Relevance: Supports citable, versioned scholarly communication."README includes SPDX-License-Identifier: CC-BY-4.0 and badges for documentation, releases, tests, Zenodo DOIs, Snakemake, Discord, and REUSE status."
Relevance: Shows open licensing and discoverability of communication, documentation, releases, and support channels."Contributing guide carries SPDX-License-Identifier: CC-BY-4.0 and describes how to build and contribute documentation."
Relevance: Shows reusable public documentation and procedures for maintaining communication materials.Open Linkage
Evidence Found
The project has moderate-to-strong linkage evidence across the chain: the README connects the model claim to named input datasets, Snakemake workflow construction, documentation, limitations, Zenodo records, releases, and papers; CITATION.cff identifies a specific version and repository; the data-versioning validation layer implies structured links among datasets, versions, sources, tags, dates, and URLs; and CI workflows connect code revisions to tests, validation, release preparation, and container images tagged by commit SHA.
Gaps Identified
However, the evidence does not show an explicit, externally inspectable provenance graph from a specific environmental claim or publication result to exact data versions, code commit, model configuration, solver environment, workflow run logs, outputs, review records, and publication artifacts. The linkage is therefore substantially present at project-infrastructure level but incomplete at exact claim/run level. It is not lower because several artifacts deliberately support versioning and cross-referencing; it is not higher because exact end-to-end claim-level traceability is not demonstrated in the supplied bundle.
"Links the model description to named datasets, Snakemake, documentation, limitations, GitHub issues, Zenodo DOIs, releases, and example papers."
Relevance: Provides project-level links among claims, inputs, methods, publications, and community channels."Specifies repository https://github.com/pypsa/pypsa-eur and version v2026.02.0 for citation."
Relevance: Links citation/publication identity to a versioned software artifact."Validates data version records with dataset, version, source, tags, added date, note, and URL, including rules for archive URLs and latest-version consistency."
Relevance: Shows explicit infrastructure for linking model data layers to versioned source/archive records."Builds or retags development container images with ghcr.io tags equal to the GitHub commit SHA."
Relevance: Links execution environments to exact code revisions, supporting verifiable chain construction.Critical Traceability Gaps
Inspection details of where the evidence chain breaks down and what that implies for public trust.
| Identified Gap | Impact on Traceability | Suggested Remediation |
|---|---|---|
| Dataset not linked or versioned | High ‐ Blocks inspection of inputs | Register raw datasets in Zenodo and cite unique DOIs in config files. |
Remediation & Improvements Roadmap
Actionable roadmap prioritized by implementation complexity.
Quick Wins
Low effort, high immediate return.
- Add CITATION.cff file
Enables clear publication-to-repository citation schema.
- Commit top-level LICENSE
Resolves legal terms of software and dataset inspectability.
- Link communication channels
Add status/discord badges to repository README files.
Medium Effort
Standard engineering practices.
- Specify configurations in config files
Provide explicit parameters and dataset configurations in repository.
- Lock computational environments
Lock dependencies using Conda-lock or Docker image digests.
- Archive automated workflow logs
Publish CI test summaries and solver objective check values.
Strategic Upgrades
Highest traceability return.
- Publish workflow runs on Zenodo
Create persistent, immutable archives of the full execution outputs.
- Build claims-to-evidence provenance graphs
Establish a machine-readable map of the end-to-end evidence trail.
Framework Dimension Definitions
- Open Input Data and Measurement Evidence: Identifiable, documented, attributable, versioned raw measurement evidence inputs.
- Open-Source Models, Methods, and Software: Inspectability of Analytical logic via code repositories, configurations, and license files.
- Open Execution and Reproducibility: Computational environments, workflows, run logs, and outputs make rerun processes inspectable.
- Open Community and Review: Scrutiny, peer review, issue tracking, and responses to replication challenges are visible.
- Open Publications and Communication: Resulting reports, papers, and dashboards are openly accessible and reference source materials.
- Open Linkage: The versioned, explicit connectivity linking data, models, execution, review, and publication steps.
Human Review Notes
Human review of this assessment report is pending. An independent reviewer must inspect the repository references to verify the findings.