Submitting results

If you’re interested in submitting results to RCMIP then you’re in the right place. Here we go through the process of preparing and submitting results to RCMIP. If you have any issues with this guide, or feel it could be improved, please don’t hesitate to raise an issue in the pyrcmip issue tracker or make a merge request.

A set of Jupyter Notebooks for the running the RCMIP experiments and uploading the results using the Geoffroy et al. (2013) two-layer model, as implemented in openscm-twolayermodel are available in notebooks/example-model-pipeline. These notebooks can be launched directly using binder. We would love to share more examples of running your models using the RCMIP protocol.

Performing the experiments
Preparing the submission
Validating the submission
Uploading the submission

Performing the experiments 

The first step to submitting is performing the experiments. Our protocol is currently available from the RCMIP website, under the initial datasets header. Please follow the protocol as closely as possible. If you have any questions about the protocol or how to follow it, please raise an issue in the pyrcmip issue tracker.

Preparing the submission 

Having performed the experiments, next you need to prepare your submission. Submission via pyrcmip is a largely automated process, hence looks a little different to how submission looked in RCMIP phase 1.

For submission via pyrcmip, you need three things:

Timeseries to be submitted
Model reported metrics
Metadata about your submission

Timeseries 

The first part of the submission is the timeseries. These can be provided in one of three ways.

As the your_data sheet in our submission protocol (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.xlsx).
As a standalone csv (or gzipped csv) of the same format as the your_data sheet in our submission protocol (https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.csv).
As a standalone netCDF file in scmdata’s netCDF format (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.csv, further details on the format at https://github.com/openscm/scmdata/blob/v0.6.3/notebooks/netcdf.ipynb).

Differences from RCMIP Phase 1 

For those who submitted to RCMIP Phase 1, please note the following two differences:

we now ask for an extra column ensemble_member, which provides an index so we can distinguish different model configurations within a probabilistic ensemble
the column headings have changed slightly (our readers should be able to handle the old style, but updating if you can would be much appreciated)

Model reported metrics 

We also ask you to report some metrics which cannot be derived from any RCMIP experiments. At this stage, the only such metric is Equilibrium Climate Sensitivity (none of our experiments are long enough to reach true equilibrium). We ask that you submit a csv which documents the Equilibrium Climate Sensitivity of each ensemble_member provided in the timeseries part of the submission. An example of such a csv is shown in https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_reported_metrics_test.csv.

Metadata 

The final part of the submission is metadata. This simply provides metadata about your model which can be used as documentation. This metadata can be provided in one of two ways:

as a csv of the same format as https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_metadata_test.csv
by saving the meta_model sheet of our submission protocol as a standalone csv (this should result in a csv like https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip-model-meta-test.csv)

Differences from RCMIP Phase 1 

We have only made one change compared to RCMIP Phase 1:

we have removed the ECS column from the meta_model sheet

Validating the submission 

Once you have prepared your submission, you can then use RCMIP’s command-line interface to validate it. This is done using the rcmip validate command. For full details, please see the validate section in our Command-line interface documentation. This command will validate your submission, highlighting any errors it finds and providing you with a green light otherwise. If your submission does not pass validation, you will not be able to upload it in the next step. If you have any questions or issues with validation, please raise an issue in the pyrcmip issue tracker.

Note

The validation and uploading process can take some time (and a lot of memory) especially with large ensembles.

If you are having issues uploading large ensembles of results, split the input timeseries into smaller, more manageable chunks and pass all those chunks to the validate or upload command. Each chunk will be processed independently.

Uploading the submission 

Once your submission has been validated, you can then upload it. This is done using the rcmip upload command. For full details, please see the upload section in our Command-line interface documentation. This command will validate (again, just in case) and then upload your submission (assuming the validation passed). If you have any questions or issues with upload, please raise an issue in the pyrcmip issue tracker.