IO API

Input and output handling

pyrcmip.io.ensure_dir_exists(fp)

Ensure directory exists

Parameters:

fp (str) – Filepath of which to ensure the directory exists

pyrcmip.io.read_results_submission(results)

Read results submission

Parameters:

results (str or list of str) – Files to read in. All files to be read should be formatted as csv or xlsx files following the formatting defined in the template spreadsheet.

Returns:

Results read in from the submission(s)

Return type:

scmdata.ScmRun

pyrcmip.io.read_submission_model_comments(fp)

Read the submission comments.

Parameters:

fp (str) – Filepath to read

Return type:

pd.DataFrame

pyrcmip.io.read_submission_model_metadata(fp)

Read the model metadata component of a submission

Parameters:

fp (str) – Filepath to read

Return type:

pd.DataFrame

pyrcmip.io.read_submission_model_reported(fp)

Read the model reported component of a submission

Parameters:

fp (str) – Filepath to read

Return type:

pd.DataFrame

pyrcmip.io.temporary_file_to_upload(df, max_size=1024, compress=False, file_format='csv')

Create a gzipped temporary serialized version of a file to upload

Attempts to keep the file in memory until it exceeds max_size. The file is then stored on-disk and cleaned up at the end of the context.

The temporary location can be overriden using the TMPDIR environment variable as per https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir

Notice that this only supports taking an scmdata.ScmRun`|:obj:`xarray.Dataset if the target format is CSV (file_format=”csv”), and xarray.Dataset if the target format is netCDF (file_format=”nc”).

Parameters:

  • df (scmdata.ScmRun | pd.DataFrame | xarray.Dataset) – Run to store

  • max_size (int or float) – Max size in MB before file is temporarily streamed to disk. Defaults to 1GB

  • compress (bool) – whether to compress the file to upload

  • file_format (str) – File format for the target file to upload. Allowed values are: “csv” and “nc”.

Returns:

Open file object ready to be streamed

Return type:

tempfile.SpooledTemporaryFile