Skip to content

Commit

Permalink
Merge pull request #2038 from jaimergp/open-gpu-server
Browse files Browse the repository at this point in the history
document gpu/long-running builds
  • Loading branch information
isuruf authored Dec 5, 2023
2 parents abb0e2e + 9a77c61 commit 025e848
Show file tree
Hide file tree
Showing 2 changed files with 31 additions and 6 deletions.
33 changes: 29 additions & 4 deletions src/maintainer/knowledge_base.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1668,6 +1668,32 @@ The steps involved are, roughly:
consider merging the PR opened at step 2 now so it can apply to all the downstream feedstocks.
.. _opengpuserver:
Packages that require a GPU or long-running builds
==================================================
conda-forge has access to `an OpenStack server <https://github.com/Quansight/open-gpu-server>`__ that provides GPU builds and long-running builds (beyond the usual 6h limit).
If your package needs a GPU to be built or tested, or its compilation times are so long that they are currently done manually off-CI, you can request access to these runners.
To do so:
1. Open a PR in `conda-forge/admin-requests <https://github.com/conda-forge/admin-requests>`__. Follow the instructions in the repository README.
Note you need to request the type of resource you want access to (e.g. GPU runners, or long-running CPU builds)
Once merged, this will enable the self-hosted Github Actions runners for your feedstock.
2. In order to trigger jobs for these runners, the maintainer must have read and agreed to the open-gpu-server `terms of use <https://github.com/Quansight/open-gpu-server/blob/main/TOS.md>`__.
You will need to open a PR in the open-gpu-server repository, as instructed in their README.
You only need to do this once per maintainer (e.g. if you maintain multiple feedstocks).
3. Finally, you can configure your feedstock to use the self-hosted runners. A PR will have been created by admin-requests after the PR in step (1) is merged.
However, due to security measurements imposed by Github, automated re-rendering is not possible when they modify Github Actions workflows.
You will need to rerender it manually by running ``conda-smithy rerender`` in your machine and then commit and push the result.
.. note::
Due to some technical and legal limitations, some of the usual automation infrastructure is not available for these runners.
As mentioned above, the conda-forge bots won't be able to rerender your feedstock automatically anymore.
Automerge will not function properly either. Also note that the conda-forge bots won't be able to trigger the self-hosted runners.
Closing and reopening the PR won't work, but a maintainer with sufficient permissions can trigger it manually by pushing an empty commit.
.. _osxarm64:
Apple Silicon builds
Expand Down Expand Up @@ -1833,12 +1859,11 @@ higher than the package uploaded with the ``dev`` label.
How to update your feedstock token?
====================================
To reset your feedstock token and fix issues with uploads, follow these steps:
1. Create a new text file in the ``token_reset`` directory of the ``conda-forge/admin-requests`` repo.
2. Add the name of your feedstock in the text file. While adding the name, don't add "-feedstock" to the end of it. For example: for ``python-feedstock``, just add ``python``.
To reset your feedstock token and fix issues with uploads, follow these steps:
See `token_reset/example.txt <https://github.com/conda-forge/admin-requests/blob/main/token_reset/example.txt>`__ for an example.
1. Go to the ``conda-forge/admin-requests`` repo and copy `examples/example-broken.yml <https://github.com/conda-forge/admin-requests/blob/main/examples/example-broken.yml>`__ to the ``requests/`` folder.
2. Add the name of your feedstock in the YML file. While adding the name, don't add "-feedstock" to the end of it. For example: for ``python-feedstock``, just add ``python``.
.. _using_arch_rebuild:
Expand Down
4 changes: 2 additions & 2 deletions src/maintainer/updating_pkgs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -255,8 +255,8 @@ If instead the actual contents of the package are broken, the following steps wi
remove broken packages from the ``main`` channel:

1. Locate the paths to broken files on `anaconda.org <https://anaconda.org>`__, by searching for the conda-forge package and switching to the files tab.
2. Fork `conda-forge/admin-requests <https://github.com/conda-forge/admin-requests>`__ and add a new text file in the ``broken`` directory.
3. Add the broken files to the new text file, one path per line. See `broken/example.txt <https://github.com/conda-forge/admin-requests/blob/main/broken/example.txt>`__ for an example file.
2. Fork `conda-forge/admin-requests <https://github.com/conda-forge/admin-requests>`__ and add a new YML file in the ``requests`` directory.
3. Add the broken files to the new YML document.See `examples/example-broken.yml <https://github.com/conda-forge/admin-requests/blob/main/examples/example-broken.yml>`__ for an example file.
4. Open a new PR. Once merged, a bot will label all listed files as broken, thus effectively removing them from the channel.


Expand Down

0 comments on commit 025e848

Please sign in to comment.