-
Notifications
You must be signed in to change notification settings - Fork 192
How to make a new release
Creating a new release consists of the following steps:
In the following, we assume that the latest release was v0.8.1.
Furthermore, we assume the following naming conventions for remotes:
-
origin: remote pointing toaiidateam/aiida_core -
fork: remote pointing to personal fork, e.g.sphuber/aiida_core
Check how your remotes are configured with git remote -v
We use semantic versioning, i.e. version labels have the form v<major>.<minor>.<patch>
- Patch release:
v0.8.1tov0.8.2, only bug fixes - Minor release:
v0.8.1tov0.9.0, bug fixes and new features that maintain backwards compatibility - Major release:
v0.8.1tov1.0.0, bug fixes and new features that break backwards compatibility
We use the gitflow branching model. In short: since the last release, new features and bug fixes will have been merged into the develop branch through pull requests.
Branch off the new release branch from the latest release tag.
git checkout tags/v0.8.1 -b release/0.8.2
Note: Usually, this branch will already exist and you can skip this step!
If develop holds bug fixes that need to go in this patch release, we need to cherry pick the corresponding pull requests and apply them to the release branch.
A patch file describes the changes to the repository as a result of a set of commits. For each pull request, Github provides a URL to download the corresponding patch file, e.g. for pull request 624:
https://patch-diff.githubusercontent.com/raw/aiidateam/aiida_core/pull/624.patch
Piping the patch file to git am:
curl https://patch-diff.githubusercontent.com/raw/aiidateam/aiida_core/pull/624.patch | git am
applies the changes introduced by pull request 624 to our current checked out branch and automatically creates a new commit.
Do this for all the pull requests that need to go in this patch release. Make sure to start at the oldest pull requests and applying the most recent pull request last in order to minimize the chances of merge conflicts.
For major and minor releases, we assume that all the changes in the current develop branch have to be included in the release. As such, branch off the new release branch directly from develop:
git fetch origin # just making sure we are up to date
git checkout origin/develop -b release/0.9.0
If develop contains changes that cannot be included in the release, you'll need to take a different approach. One option is to follow the approach outlined above for the patch release, i.e. cherry-picking individual pull requests.
Once you've prepared the release branch locally, push it to Github
git push origin release/0.9.0
Then, protect it from untested merges: On Github, navigate to Settings => Branches and select your branch. Check the boxes that require any pull requests to pass the tests.
From this point onwards, any further bug fixes or features for the release need to be made via a pull request to the release branch.
Go through the pull requests and updating the CHANGELOG.md file with all significant changes made.
Note: Save yourself some work by using the script API_clients/github_pr_for_release.py from the private aiida_helpers repository.
Open the script, read the header, change the few variables at the top of the file, and then run it in order to produce a MarkDown-formatted list of issues with links etc, where the content is the title and body of the PR.
All that remains is to go through the list and clean it up.
Check pull requests merged into the release branch (filter by is:pr base:release/0.8.2)
Check pull requests merged into develop, since the release branch of the last minor (not bugfix) release was branched off:
- Find out, when last release was branched off
git show --summary `git merge-base release/0.8.0 develop`
or, if that doesn't work:git show v0.8.0 - Use date to filter pull requests:
is:pr merged:>2017-10-18 base:develop
NOTE: the above filter on base: is not fully correct: it will not show PRs that merged in a branch during the PR, and for which that branch was merged later into develop. Rather, use the script inside the aiida_helpers repository, described a few lines above.
If a pull request introduces or changes functionality, it should be documented. If documentation is missing, ask the author of the pull request to provide it.
Note: This can take time, but it's a straightforward task, where you can easily ask for help from others.
In order to update the AUTHORS.txt, check for authors on Insights => Contributors and select the appropriate time window.
If the layout of the database or of configuration files changed, a release needs to include corresponding migration scripts.
Make sure these scripts are included and documented, and manually test the scripts.
Check out the release branch in your repository and make sure you have loaded a virtual environment with python 2.7.
git clone [email protected]:aiidateam/aiida_helpers
python aiida_helpers/helpers/clean_old_headers.py -b <release_branch_name> -d docs tests -- <aiida_core_path>
python aiida_helpers/helpers/add_copyright_notice.py -b <release_branch_name> -d docs -- <aiida_core_path>
Note: We explicitly exclude the docs directory as it contains a lot of small python files with code snippets and we do not want them to contain the copyright notice.
Note: If no new files were added, it is possible that these scripts do not result in any code changes.
Finally, update the source code version in aiida/__init__.py by hand.
Commit and push -- your branch is now ready to be released!
Now that the release branch is ready, merge it back into master via a pull request:
Make sure the release/0.9.0 branch in your fork is up to date, go to Github and create a pull request to the master branch of the official repository named Release version v0.9.0
(Note: you can also create the PR from the release/0.9.0 branch of the official repository, if you have access to it).
After the pull request has been approved and merged, pull the changes into your local master branch, tag the final merge commit and push it to the remote:
git pull origin master
git tag -a v0.9.0 -m 'v0.9.0'
git push --tags origin master
If you accidentally tagged the wrong commit, you can delete the local and remote tags using the following commands.
git tag -d v0.9.0
git push --delete origin v0.9.0
We now need to merge master back into develop, however, do not do this directly through a pull request from master to develop. This is because there is a branch protection on develop that prevents pull requests from branches which are not up to date (this would mean that you would first have to merge develop into master). Instead, create a new temporary branch from master, into which you merge develop and then create a pull request from this new branch back into develop. In the following example we assume that the remote pointing to the main repository is labeled origin, otherwise replace it with the appropriate label.
git checkout -b merge/master origin/develop
git merge master
git push origin merge/master
After this, create a pull request from the newly created branch into develop. When the pull request is merged delete the release and the merging branch. Note that you may have to remove the branch protection from the release branch first on Github.
git push origin :release/1.5.0
git push origin :merge/master
With the release tag created, the new release can be built and uploaded to the package repository.
After a release has been made, the first commit that should be made to develop is to add the .post0 qualifier to the version attribute in the aiida/__init__.py.
An example can be seen in this commit.
This will warn users that are (accidentally) running of the develop branch in order to prevent they perform production runs with a non-released version:
Warning: You are currently using a post release development version of AiiDA: 2.0.0.post0
Warning: Be aware that this is not recommended for production and is not officially supported.
Warning: Databases used with this version may not be compatible with future releases of AiiDA
Warning: as you might not be able to automatically migrate your data.
This works because every time Manager.load_profile is called, it calls Manager.check_version which checks the version attribute of the current installation, and if it is a post-release (i.e. the version attribute has the post release qualifier), the warning is displayed. The choice for using the post-release identifier is a bit arbitrary, but we chose it because it is PEP440 compliant which allow us to reliably use the version parsing functionality of the packaging library.
RTD is set to automatically activate SemVer versions: https://readthedocs.org/dashboard/aiida-core/rules/, but it is best to check that this has occured.
- PyPI build: There is a
releaseGitHub Action that should trigger when a GitHub release (and hence git tag is created), to deploy the latest version to PyPI. Check in https://github.com/aiidateam/aiida-core/actions/workflows/release.yml that this action triggered and successfully completed. For reference, the old instructions are included below - Some time after the PyPI release is uploaded, the Conda-forge bot should automatically create a PR for the new release: this does not automatically update dependencies! So check those, then merge.
- Update dependent repositories:
- aiida-metapkg (also has travis hook for PyPI deployment)
- aiida-docker-base
- aiida-docker-stack
- aiida_docker_compose
- aiidalab-metapkg (also has travis hook for PyPI deployment)
- Update aiida.net
- Download page: provide links to the changelog, github and a download link (via github) pointing to the release tag. Update the list of previous releases.
- Add a news entry referencing the new release.
- Announcements
- Send a new message on Mailinglist
This should no longer be necessary! (see above)
To ensure we build from a clean repository, clone a new copy of the repository and checkout the latest release
git clone [email protected]:aiidateam/aiida-core release
cd release
git checkout tags/v0.9.0 -b v0.9.0
Create a new virtual environment to run the build in. For example if you have the virtualenvwrapper:
mkvirtualenv aiida_release
pip install filt
Then, create the build and upload it:
flit publish
Verify that the release build was successfully uploaded by installing it in a fresh virtual environment
mkvirtualenv aiida_test_release
pip install aiida-core
If a fix needs to be available urgently, it makes sense to create a patch release containing just the critical change by cherry-pick, and keeping the rest of the changes for the next minor/major release. Assuming the current release is v2.0.1, the following steps need to be followed in order to make a patch release:
- Make a new branch from the last release:
git checkout v2.0.1 -b release/2.0.2 - Cherry-pick the critical commit(s) to this branch:
git cherry-pick <commit-hash> - Amend the commit message to include the original commit hash:
git commit --amend
and append"Cherry-pick: <original-commit-hash>"to the end - Update the version number in
aiida/__init__.py - Update
CHANGELOG.md git commit -m "Release `v2.0.2`"- Push the branch to GitHub, wait for tests to succeed, and ask other maintainers to review the branch
- Make a release and tag on GitHub based on this branch. PyPi is updated automatically.
- Continue locally by merging
release/2.0.2tomain, and solve the version number conflict by using the new version (2.0.2) - In main branch, update version number in
aiida/__init__.pyto2.0.2.post0and commit by
git commit -m "Post release: add the `.post0` qualifier to version attribute"
and push to GitHub. - The
release/2.0.2branch can be deleted both locally and on GitHub.
Note: Unfortunately there is no good way for other maintainers to review and comment on this process e.g. via a PR.