pyrcmip¶
pyrcmip is a tool for validating and uploading results to RCMIP. The Reduced Complexity Model Intercomparison Project (RCMIP) is a project to evaluate reduced-complexity (also known as simple) climate models and compare them against CMIP coupled models.
License¶
pyrcmip is free software under a BSD 3-Clause License, see LICENSE.
If you make use of pyrcmip or any of the RCMIP project, please cite Nicholls et al., GMDD 2020 [1].
References¶
- 1
Z. R. J. Nicholls, M. Meinshausen, J. Lewis, R. Gieseke, D. Dommenget, K. Dorheim, C.-S. Fan, J. S. Fuglestvedt, T. Gasser, U. Golüke, P. Goodwin, E. Kriegler, N. J. Leach, D. Marchegiani, Y. Quilcaille, B. H. Samset, M. Sandstad, A. N. Shiklomanov, R. B. Skeie, C. J. Smith, K. Tanaka, J. Tsutsui, and Z. Xie. Reduced complexity model intercomparison project phase 1: protocol, results and initial observations. Geoscientific Model Development Discussions, 2020:1–33, 2020. URL: https://gmd.copernicus.org/preprints/gmd-2019-375/, doi:10.5194/gmd-2019-375.
Installation¶
The easiest way to install pyrcmip is with pip. At this stage pyrcmip only supports Python 3.6+.
# if you're using a virtual environment, make sure you're in it
pip install pyrcmip
Submitting results¶
If you’re interested in submitting results to RCMIP then you’re in the right place. Here we go through the process of preparing and submitting results to RCMIP. If you have any issues with this guide, or feel it could be improved, please don’t hesitate to raise an issue in the pyrcmip issue tracker or make a merge request.
A set of Jupyter Notebooks for the running the RCMIP experiments and uploading the results using the Geoffroy et al. (2013) two-layer model, as implemented in openscm-twolayermodel are available in notebooks/example-model-pipeline. These notebooks can be launched directly using binder. We would love to share more examples of running your models using the RCMIP protocol.
Performing the experiments¶
The first step to submitting is performing the experiments. Our protocol is currently available from the RCMIP website, under the initial datasets header. Please follow the protocol as closely as possible. If you have any questions about the protocol or how to follow it, please raise an issue in the pyrcmip issue tracker.
Preparing the submission¶
Having performed the experiments, next you need to prepare your submission. Submission via pyrcmip is a largely automated process, hence looks a little different to how submission looked in RCMIP phase 1.
For submission via pyrcmip, you need three things:
Timeseries to be submitted
Model reported metrics
Metadata about your submission
Timeseries¶
The first part of the submission is the timeseries. These can be provided in one of three ways.
As the
your_datasheet in our submission protocol (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.xlsx).As a standalone csv (or gzipped csv) of the same format as the
your_datasheet in our submission protocol (https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.csv).As a standalone netCDF file in scmdata’s netCDF format (e.g. https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_output_test.csv, further details on the format at https://github.com/openscm/scmdata/blob/v0.6.3/notebooks/netcdf.ipynb).
Differences from RCMIP Phase 1¶
For those who submitted to RCMIP Phase 1, please note the following two differences:
we now ask for an extra column
ensemble_member, which provides an index so we can distinguish different model configurations within a probabilistic ensemblethe column headings have changed slightly (our readers should be able to handle the old style, but updating if you can would be much appreciated)
Model reported metrics¶
We also ask you to report some metrics which cannot be derived from any RCMIP experiments.
At this stage, the only such metric is Equilibrium Climate Sensitivity (none of our experiments are long enough to reach true equilibrium).
We ask that you submit a csv which documents the Equilibrium Climate Sensitivity of each ensemble_member provided in the timeseries part of the submission.
An example of such a csv is shown in https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_reported_metrics_test.csv.
Metadata¶
The final part of the submission is metadata. This simply provides metadata about your model which can be used as documentation. This metadata can be provided in one of two ways:
as a csv of the same format as https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip_model_metadata_test.csv
by saving the
meta_modelsheet of our submission protocol as a standalone csv (this should result in a csv like https://gitlab.com/rcmip/pyrcmip/-/tree/master/tests/data/rcmip-model-meta-test.csv)
Differences from RCMIP Phase 1¶
We have only made one change compared to RCMIP Phase 1:
we have removed the ECS column from the
meta_modelsheet
Validating the submission¶
Once you have prepared your submission, you can then use RCMIP’s command-line interface to validate it.
This is done using the rcmip validate command.
For full details, please see the validate section in our Command-line interface documentation.
This command will validate your submission, highlighting any errors it finds and providing you with a green light otherwise.
If your submission does not pass validation, you will not be able to upload it in the next step.
If you have any questions or issues with validation, please raise an issue in the pyrcmip issue tracker.
Uploading the submission¶
Once your submission has been validated, you can then upload it.
This is done using the rcmip upload command.
For full details, please see the upload section in our Command-line interface documentation.
This command will validate (again, just in case) and then upload your submission (assuming the validation passed).
If you have any questions or issues with upload, please raise an issue in the pyrcmip issue tracker.
Development¶
If you’re interested in contributing to pyrcmip, we’d love to have you on board! This section of the docs details how to get setup to contribute and how best to communicate.
Contributing¶
All contributions are welcome, some possible suggestions include:
tutorials (or support questions which, once solved, result in a new tutorial :D)
blog posts
improving the documentation
bug reports
feature requests
pull requests
Please report issues or discuss feature requests in the pyrcmip issue tracker. If your issue is a feature request or a bug, please use the templates available, otherwise, simply open a normal issue :)
As a contributor, please follow a couple of conventions:
Create issues in the pyrcmip issue tracker for changes and enhancements, this ensures that everyone in the community has a chance to comment
Be welcoming to newcomers and encourage diverse new contributors from all backgrounds: see the Python Community Code of Conduct
Getting setup¶
To get setup as a developer, we recommend the following steps (if any of these tools are unfamiliar, please see the resources we recommend in Development tools):
Install conda and make
Run
make conda-environment, if that fails you can try doing it manually by reading the commands from theMakefileMake sure the tests pass by running
make test, as above if that fails you can try doing it manually by reading the commands from theMakefile
Getting help¶
Whilst developing, unexpected things can go wrong (that’s why it’s called ‘developing’, if we knew what we were doing, it would already be ‘developed’). Normally, the fastest way to solve an issue is to contact us via the issue tracker. The other option is to debug yourself. For this purpose, we provide a list of the tools we use during our development as starting points for your search to find what has gone wrong.
Development tools¶
This list of development tools is what we rely on to develop pyrcmip reliably and reproducibly. It gives you a few starting points in case things do go inexplicably wrong and you want to work out why. We include links with each of these tools to starting points that we think are useful, in case you want to learn more.
- Conda virtual environments
note the common gotcha that
source activatehas now changed toconda activatewe use conda instead of pure pip environments because they help us deal with Iris’ dependencies: if you want to learn more about pip and pip virtual environments, check out this introduction
- Continuous integration (CI)
we use GitLab CI for our CI but there are a number of good providers
- Jupyter Notebooks
we’d recommend simply installing
jupyter(conda install jupyter) in your virtual environment
Other tools¶
We also use some other tools which aren’t necessarily the most familiar. Here we provide a list of these along with useful resources.
- Regular expressions
we use regex101.com to help us write and check our regular expressions, make sure the language is set to Python to make your life easy!
Formatting¶
To help us focus on what the code does, not how it looks, we use a couple of automatic formatting tools.
These automatically format the code for us and tell use where the errors are.
To use them, after setting yourself up (see Getting setup), simply run make black and make flake8.
Note that make black can only be run if you have committed all your work i.e. your working directory is ‘clean’.
This restriction is made to ensure that you don’t format code without being able to undo it, just in case something goes wrong.
Buiding the docs¶
After setting yourself up (see Getting setup), building the docs is as simple as running make docs (note, run make -B docs to force the docs to rebuild and ignore make when it says ‘… index.html is up to date’).
This will build the docs for you.
You can preview them by opening docs/build/html/index.html in a browser.
For documentation we use Sphinx. To get ourselves started with Sphinx, we started with this example then used Sphinx’s getting started guide.
Gotchas¶
To get Sphinx to generate pdfs (rarely worth the hassle), you require Latexmk.
On a Mac this can be installed with sudo tlmgr install latexmk.
You will most likely also need to install some other packages (if you don’t have the full distribution).
You can check which package contains any missing files with tlmgr search --global --file [filename].
You can then install the packages with sudo tlmgr install [package].
Docstring style¶
For our docstrings we use numpy style docstrings. For more information on these, here is the full guide and the quick reference we also use.
Releasing¶
The steps to release a new version of pyrcmip are shown below. Please do all the steps below and all the steps for both release platforms.
First step¶
Test installation with dependencies
make test-installUpdate
CHANGELOG.rst:add a header for the new version between
masterand the latest bullet pointthis should leave the section underneath the master header empty
git add .git commit -m "Prepare for release of vX.Y.Z"git tag vX.Y.ZTest version updated as intended with
make test-install
PyPI¶
If uploading to PyPI, do the following (otherwise skip these steps)
make publish-on-testpypiGo to test PyPI and check that the new release is as intended. If it isn’t, stop and debug.
Test the install with
make test-testpypi-install(this doesn’t test all the imports as most required packages are not on test PyPI).
Assuming test PyPI worked, now upload to the main repository
make publish-on-pypiGo to pyrcmip’s PyPI and check that the new release is as intended.
Test the install with
make test-pypi-install(a pip only install will throw warnings about Iris not being installed, that’s fine).
Conda¶
Note: Conda releases are not yet operational
If you haven’t already, fork the pyrcmip conda feedstock. In your fork, add the feedstock upstream with
git remote add upstream https://github.com/conda-forge/pyrcmip-feedstock(upstreamshould now appear in the output ofgit remote -v)Update your fork’s master to the upstream master with:
git checkout mastergit fetch upstreamgit reset --hard upstream/master
Create a new branch in the feedstock for the version you want to bump to.
Edit
recipe/meta.yamland update:version number in line 1 (don’t include the ‘v’ in the version tag)
the build number to zero (you should only be here if releasing a new version)
update
sha256in line 9 (you can get the sha from pyrcmip’s PyPI by clicking on ‘Download files’ on the left and then clicking on ‘SHA256’ of the.tar.gzfile to copy it to the clipboard)
git add .git commit -m "Update to vX.Y.Z"git pushMake a PR into the pyrcmip conda feedstock
If the PR passes (give it at least 10 minutes to run all the CI), merge
Check https://anaconda.org/conda-forge/pyrcmip to double check that the version has increased (this can take a few minutes to update)
Why is there a Makefile in a pure Python repository?¶
Whilst it may not be standard practice, a Makefile is a simple way to automate general setup (environment setup in particular).
Hence we have one here which basically acts as a notes file for how to do all those little jobs which we often forget e.g. setting up environments, running tests (and making sure we’re in the right environment), building docs, setting up auxillary bits and pieces.
Why did we choose a BSD 2-Clause License?¶
We want to ensure that our code can be used and shared as easily as possible. Whilst we love transparency, we didn’t want to force all future users to also comply with a stronger license such as AGPL. Hence the choice we made.
We recommend Morin et al. 2012 for more information for scientists about open-source software licenses.
Command-line interface¶
rcmip¶
Command-line interface for pyrcmip
rcmip [OPTIONS] COMMAND [ARGS]...
Options
-
--log-level<log_level>¶ - Options
DEBUG|INFO|WARNING|ERROR|EXCEPTION|CRITICAL
upload¶
Validate and upload data to RCMIP’s S3 bucket.
All the files for a given version have to be uploaded together.
One or more TIMESERIES files in which the timeseries output is stored. These should be
CSV or NetCDF files conforming to the format expected by scmdata. Multiple
timeseries inputs can be specified, but care must be taken to ensure that all of
the individual timeseries have unique metadata.
MODEL_REPORTED is the CSV file in which the model reported metrics are stored.
METADATA is the CSV file in which the metadata output is stored.
rcmip upload [OPTIONS] TIMESERIES... MODEL_REPORTED METADATA
Options
-
--token<token>¶ Required Authentication token. Contact zebedee.nicholls@climate-energy-college.org for a token
-
--bucket<bucket>¶
-
--model<model>¶ Required
-
--version<version>¶ Required Version of the data being uploaded. Must be a valid semver version string (https://semver.org/). For example 2.0.0
Arguments
-
TIMESERIES¶ Required argument(s)
-
MODEL_REPORTED¶ Required argument
-
METADATA¶ Required argument
validate¶
Validate submission input
Three different types of input data are required for validation, namely:
One or more TIMESERIES files in which the timeseries output is stored. These should be
CSV or NetCDF files conforming to the format expected by scmdata. Multiple
timeseries inputs can be specified, but care must be taken to ensure that all of
the individual timeseries have unique metadata.
MODEL_REPORTED is the CSV file in which the model reported metrics are stored.
METADATA is the CSV file in which the metadata output is stored.
rcmip validate [OPTIONS] TIMESERIES... MODEL_REPORTED METADATA
Arguments
-
TIMESERIES¶ Required argument(s)
-
MODEL_REPORTED¶ Required argument
-
METADATA¶ Required argument
Database API¶
Database of results handling
-
class
pyrcmip.database.Database(root_dir)¶ On-disk database handler for outputs from SCMs
-
get_out_filepath(climate_model, variable, region, scenario, ensemble_member=None)¶ Get filepath in which data has been saved
The filepath is the root directory joined with the other information provided. The filepath is also cleaned to remove spaces and special characters.
- Parameters
- Returns
Path in which to save the data. If
ensemble_memberisNonethen it is not included in the filename.- Return type
-
load_data(variable, region, scenario)¶ Load data from the database
-
load_model_reported()¶ Load all model reported results
- Returns
All model reported results
- Return type
pd.DataFrame
-
load_summary_tables()¶ Load all summary tables
- Returns
All summary tables
- Return type
pd.DataFrame
-
save_condensed_file(scmrun)¶ Save results which have multiple ensemble members
- Parameters
scmrun (
scmdata.ScmRun) – Results to save in the database- Raises
AssertionError –
ensemble_memberis not included inscmrun’s metadata
-
save_model_reported(res, key='all')¶ Save model reported data into the database
- Parameters
res (
pd.DataFrame) – Model reported results to savekey (str) – Identifier to use in the filename
- Raises
AssertionError – The columns of res are not as expected (i.e.
{"value", "ensemble_member", "RCMIP name", "unit", "climate_model"}) or more than one climate model is included inres.
-
save_summary_table(res, file_id)¶ Save summary table
- Parameters
res (
pd.DataFrame) – Summary table to savefile_id (str) – Identifier to use in the filename
- Raises
AssertionError – Columns of
resare not as expected (i.e. not equal to{"assessed_range_label", "assessed_range_value", "climate_model", "climate_model_value", "metric", "percentage_difference", "unit"})
-
save_to_database(scmrun)¶ Save a set of results to the database
The results are saved with one file for each
["climate_model", "variable", "region", "scenario", "ensemble_member"]combination.- Parameters
scmrun (
scmdata.ScmRun) – Results to save
-
-
pyrcmip.database.time_mean(self, variable=None)¶ Take time mean of data
TODO: move to scmdata and make method of
ScmRun.- Parameters
variable (str) – Value of the variable column in the output. If
None, the values are not altered from those inself.- Returns
Time mean of the data
- Return type
pd.Series
Errors API¶
Custom errors defined within pyrcmip
-
exception
pyrcmip.errors.ProtocolConsistencyError¶ Inconsistency between input data and the RCMIP protocol
IO API¶
Input and output handling
-
pyrcmip.io.ensure_dir_exists(fp)¶ Ensure directory exists
- Parameters
fp (str) – Filepath of which to ensure the directory exists
-
pyrcmip.io.read_results_submission(results)¶ Read results submission
- Parameters
results (str or list of str) – Files to read in. All files to be read should be formatted as csv or xlsx files following the formatting defined in the template spreadsheet.
- Returns
Results read in from the submission(s)
- Return type
scmdata.ScmRun
-
pyrcmip.io.read_submission_model_metadata(fp)¶ Read the model metadata component of a submission
- Parameters
fp (str) – Filepath to read
- Returns
- Return type
pd.DataFrame
-
pyrcmip.io.read_submission_model_reported(fp)¶ Read the model reported component of a submission
- Parameters
fp (str) – Filepath to read
- Returns
- Return type
pd.DataFrame
-
pyrcmip.io.temporary_file_to_upload(df, max_size=1024, compress=False)¶ Create a gzipped temporary serialized version of a file to upload
Attempts to keep the file in memory until it exceeds max_size. The file is then stored on-disk and cleaned up at the end of the context.
The temporary location can be overriden using the TMPDIR environment variable as per https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
- Parameters
- Returns
Open file object ready to be streamed
- Return type
Metric Calculations API¶
Metric calculations used in RCMIP
-
pyrcmip.metric_calculations.calculate_transient_climate_response(assessed_ranges, res_calc, norm_period, evaluation_period, unit)¶ Calculate the transient climate response (TCR)
- Parameters
assessed_ranges (
pyrcmip.assessed_ranges.AssessedRanges) – Assessed ranges instanceres_calc (
scmdata.ScmRun) – Results from which the TCR is to be derivednorm_period (list) – Years to use for normalising the data before calculating the TCR
evaluation_period (list) – Years to use when evaluating the TCR
unit (str) – Unit in which the TCR should be returned
- Returns
TCR with other relevant model metadata
- Return type
pd.DataFrame- Raises
NotImplementedError – The evaluation period is not the year 1920 i.e. the experiment does not match the CMIP6 specifications
Plotting API¶
Helpers and config for plotting
-
pyrcmip.plotting.CLIMATE_MODEL_PALETTE= {'AR6 Prelim. FGD': 'tab:gray', 'HadCRUT.5.0.0.0': 'tab:gray', 'HadCRUT.5.0.0.0 (GMST)': 'tab:gray', 'MAGICC7': 'tab:orange', 'Raw CMIP6 multi-model ensemble': 'tab:green', 'assessed range': 'tab:blue', 'two_layer': 'tab:pink', 'von Shuckmann et al. 2020': 'tab:purple'}¶ Colour palette used for plots coloured by climate model
- Type
-
pyrcmip.plotting.CMIP6_NAME= 'Raw CMIP6 multi-model ensemble'¶ String used to represent the CMIP6 multi-model ensemble in plots
- Type
-
pyrcmip.plotting.SCENARIO_PALETTE= {'historical': 'tab:gray', 'ssp119': array([0.1171875, 0.5859375, 0.515625 ]), 'ssp126': array([0.11328125, 0.19921875, 0.328125 ]), 'ssp245': array([0.9140625 , 0.86328125, 0.23828125]), 'ssp370': array([0.9453125 , 0.06640625, 0.06640625]), 'ssp370-lowNTCF': array([0.9453125 , 0.06640625, 0.06640625]), 'ssp434': array([0.38671875, 0.73828125, 0.89453125]), 'ssp460': array([0.90625 , 0.921875 , 0.19140625]), 'ssp534-over': array([0.6015625 , 0.42578125, 0.78515625]), 'ssp585': array([0.515625 , 0.04296875, 0.1328125 ])}¶ Colour palette used for plots coloured by scenario
- Type
Stats API¶
Statistics required for RCMIP analysis
-
pyrcmip.stats.get_skewed_normal(median, lower, upper, conf, input_data)¶ Get skewed normal distribution matching the inputs
- Parameters
median (float) – Median of the output distribution
lower (float) – Lower bound of the confidence interval
upper (float) – Upper bound of the confidence interval
conf (float) – Confidence associated with the interval [lower, upper] e.g. 0.66 would mean that [lower, upper] defines the 66% confidence range
input_data (
np.ndarray) – Points from the derived distribution to return. For each point, Y, ininput_data, we determine the value at which a cumulative probability of Y is achieved. As a result, all values ininput_datamust be in the range [0, 1]. Hence if you want a random sample from the derived skewed normal, simply makeinput_dataequal to a random sample of the uniform distribution [0, 1]
- Returns
Points sampled from the derived skewed normal distribution based on
input_data- Return type
np.ndarray
Validate API¶
Validation of RCMIP submissions
-
pyrcmip.validate.convert_units_to_rcmip_units(submission, protocol_variables)¶ Convert units to RCMIP units
- Parameters
submission (
scmdata.ScmRun) – Submission to convertprotocol_variables (
pd.DataFrame) – Variables and units as defined by the RCMIP protocol
- Returns
Submission with units converted to RCMIP units
- Return type
scmdata.ScmRun- Raises
ProtocolConsistencyError – Units could not be converted to RCMIP units
-
pyrcmip.validate.validate_regions(regions_to_check, protocol_regions)¶ Validate regions against regions in the RCMIP protocol
- Parameters
regions_to_check (list-like) – Regions to check
protocol_regions (list-like) – Regions in the RCMIP protocol
- Raises
ProtocolConsistencyError –
regions_to_checkcontains regions not included inprotocol_regions
-
pyrcmip.validate.validate_scenarios(scenarios_to_check, protocol_scenarios)¶ Validate scenarios against scenarios in the RCMIP protocol
- Parameters
scenarios_to_check (list-like) – Scenarios to check
protocol_scenarios (list-like) – Scenarios in the RCMIP protocol
- Raises
ProtocolConsistencyError –
scenarios_to_checkcontains scenarios not included inprotocol_scenarios
-
pyrcmip.validate.validate_submission(submission, protocol=None)¶ Validate that an RCMIP submission complies with the required data format
- Parameters
submission (
scmdata.ScmRun) – Data to validateprotocol (str) – Data file containing the RCMIP protocol against which to validate the data. If
None, the submission template will be loaded frompyrcmip/data/rcmip-data-submission-template-v4-0-0.xlsx.
- Returns
Input data, converted to match RCMIP units
- Return type
scmdata.ScmRun- Raises
ProtocolConsistencyError – The data is not consistent with the protocol
-
pyrcmip.validate.validate_submission_bundle(timeseries, model_reported, metadata, protocol=None)¶ Validate that an RCMIP submission bundle complies with the required formats
- Parameters
timeseries (
scmdata.ScmRun) – Timeseries to validatemodel_reported (
pd.DataFrame) – Model reported metricsmetadata (
pd.DataFrame) – Model metadataprotocol (str) – Data file containing the RCMIP protocol against which to validate the timeseries. If
None, the submission template will be loaded frompyrcmip/data/rcmip-data-submission-template-v4-0-0.xlsx.
- Returns
Validated timeseries, model reported metrics and model metadata
- Return type
(
scmdata.ScmRun,pd.DataFrame,pd.DataFrame)- Raises
ProtocolConsistencyError – The submission bundle is not consistent with the RCMIP protocol
ValueError – A value for
climate_modelis found intimeseriesormodel_reportedbut isn’t found in theclimate_modelcolumn ofmetadata.
-
pyrcmip.validate.validate_submission_model_meta(inp)¶ Validate a submission’s metadata
- Parameters
inp (
pd.DataFrame) – Metadata submission to validate- Returns
Validated metadata submission
- Return type
pd.DataFrame- Raises
ProtocolConsistencyError – The columns of res are not as expected (i.e.
{"climate_model", "climate_model_name", "climate_model_version", "climate_model_configuration_label", "climate_model_configuration_description", "project", "name_of_person", "literature_reference"}).
-
pyrcmip.validate.validate_submission_model_reported_metrics(inp)¶ Validate a submission of model reported metrics
- Parameters
inp (
pd.DataFrame) – Input to validate- Returns
Validated input
- Return type
pd.DataFrame- Raises
ProtocolConsistencyError – The columns of res are not as expected (i.e.
{"value", "ensemble_member", "RCMIP name", "unit", "climate_model"}), more than one climate model is included inres, theensemble_membercolumn is not integers, an unrecognised metric is provided or the provided unit is not compatible with RCMIP.
-
pyrcmip.validate.validate_variables(vars_to_check, protocol_variables)¶ Validate variables against variables in the RCMIP protocol
- Parameters
vars_to_check (list-like) – Variables to check
protocol_variables (list-like) – Variables in the RCMIP protocol
- Raises
ProtocolConsistencyError –
vars_to_checkcontains variables not included inprotocol_variables
Changelog¶
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
The changes listed in this file are categorised as follows:
Added: new features
Changed: changes in existing functionality
Deprecated: soon-to-be removed features
Removed: now removed features
Fixed: any bug fixes
Security: in case of vulnerabilities.