# TODO alex: update this file with definitive links.

Submitting results

If you’re interested in submitting results to RCMIP then you’re in the right place. Here we go through the process of preparing and submitting results to RCMIP. If you have any issues with this guide, or feel it could be improved, please don’t hesitate to raise an issue in the pyrcmip issue tracker or make a merge request.

A set of Jupyter Notebooks exemplifying the running of RCMIP experiments and uploading the results using the Geoffroy et al. (2013) two-layer model, as implemented in openscm-twolayermodel are available in notebooks/example-model-pipeline. These notebooks can be launched directly using binder. We would love to share more examples of running your models using the RCMIP protocol.

https://mybinder.org/badge_logo.svg

Performing the experiments

The first step to submitting is performing the experiments. Our protocol is currently available from the RCMIP website, under the Phase 3 header in Phases, manuscripts and data. Please follow the protocol as closely as possible. If you have any questions about the protocol or how to follow it, please raise an issue in the pyrcmip issue tracker.

Preparing the submission

Having performed the experiments, next you need to prepare your submission. Submission via pyrcmip is a largely automated process, although it is a little different compared to previous RCMIP phases.

For submission via pyrcmip, you need at least two things:

  1. Timeseries to be submitted

  2. Metadata about your submission

Optionally, you can include an additional file with comments about your submission.

Timeseries

The first part of the submission is the timeseries. These can be provided in one of three ways. TODO: updates these links when merging into orginal repo.

  1. As the your_data sheet in our submission protocol (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_output_test.xlsx).

  2. As a standalone csv (or gzipped csv) of the same format as the your_data sheet in our submission protocol (https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_output_test.csv). Notebooks 011_cleaning_the_model_output.ipynb and 021_submitting_from_csv.ipynb show how this can be done.

  3. As a standalone netCDF file in scmdata’s netCDF format (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_output_test.nc, further details on the format at https://github.com/openscm/scmdata/blob/v0.6.3/notebooks/netcdf.ipynb). Notebooks 011_cleaning_the_model_output.ipynb and 020_submitting_from_netcdf.ipynb show how this can be done.

Differences from RCMIP Phase 2

#. We now upload model submissions in netCDF format, rather than in CSV format. This change should be transparent to the user, automatically handled by the tooling. #. As a consequence of the change above, downloading previous submissions will download them in netCDF format. This tooling has been enhanced to allow users to translate these netCDF files back to CSV files, if desired, with the rcmip convert-to-csvs command. #. We are not requesting the reporting of any model metrics like in previous rounds. Instead, we are requesting full ensemble submissions, which should enable the calculation of all main climate metrics (if and when desired).

Differences from RCMIP Phase 1

For those who submitted to RCMIP Phase 1, please note the following two differences:

  1. We now ask for an extra column ensemble_member, which provides an index so we can distinguish different model configurations within a probabilistic ensemble

  2. The column headings have changed slightly (our readers should be able to handle the old style, but updating if you can would be much appreciated)

Metadata

The second part of the submission is metadata. This simply provides metadata about your model which can be used as documentation. Notice that submissions must be model-specific, and metadata is therefore only describing a single model per submission. This metadata can be provided in one of three ways:

  1. As the meta_model sheet in our submission protocol (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_output_test.xlsx).

  2. As a csv of the same format as https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_metadata_test.csv

  3. by saving the meta_model sheet of our submission protocol as a standalone csv (this should result in a csv like https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3-model-meta-test.csv)

Differences from RCMIP Phase 2

We have enahanced the tooling to also accept directly the excel file as a potential source of metadata. The tooling will then look for the metada in the meta_model sheet as in our submission protocol.

Differences from RCMIP Phase 1

We have only made one change compared to RCMIP Phase 1 (and none compared to RCMIP phase 2):

  1. we have removed the ECS column from the meta_model sheet

Comments

An optional third part of the submission is comments associated with the submission. This is new for RCMIP Phase 3, and can be supplied via the --comments flag. These can be provided in one of two ways. TODO: updates these links when merging into orginal repo.

  1. As the comments sheet in our submission protocol (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_output_test.xlsx).

  2. As a csv of the same format as https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3_model_comments_test.csv

  3. by saving the comments sheet of our submission protocol as a standalone csv (this should result in a csv like https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip3-model-comments-test.csv)

Validating the submission

Once you have prepared your submission, you can then use RCMIP’s command-line interface to validate it. This is done using the rcmip validate command. For full details, please see the validate section in our Command-line interface documentation. This command will validate your submission, highlighting any errors it finds and providing you with a green light otherwise. If your submission does not pass validation, you will not be able to upload it in the next step. If you have any questions or issues with validation, please raise an issue in the pyrcmip issue tracker.

Note

The validation and uploading process can take some time (and a lot of memory) especially with large ensembles.

If you are having issues uploading large ensembles of results, split the input timeseries into smaller, more manageable chunks and pass all those chunks to the validate or upload command. Each chunk will be processed independently. Make sure all timeseries belonging to the same simulation scenario are present in only one chunk.

Uploading the submission

Once your submission has been validated, you can then upload it. This is done using the rcmip upload command. For full details, please see the upload section in our Command-line interface documentation. This command will validate (again, just in case) and then upload your submission (assuming the validation passed). If you have any questions or issues with upload, please raise an issue in the pyrcmip issue tracker.