diff --git a/README.md b/README.md index 9c2bb5b..56eb1e2 100644 --- a/README.md +++ b/README.md @@ -10,16 +10,81 @@ The process of building the container image, running the tests, generating and d The report, deployed as a Github Page website can be viewed @ https://hbpmedical.github.io/pytest-mip/. +## Content -## How to run `pytest-mip` locally +* [Under the Hood](#under-the-hood) +* [Installation instructions](#installation-instructions) +* [How to run `pytest-mip` using the Docker image](#how-to-run-pytest-mip-using-the-docker-image) +* [Want to add tests for a new MIP federation?](#want-to-add-tests-for-a-new-mip-federation) +* [Funding](#funding) -### Installation instructions +## Under the Hood + +This project contains the following files and folders: + +- ``requirements.txt``: List all the Python dependencies for the `pytest-mip` project. + +- ``Dockerfile``: Contain all the commands to assemble the Docker container image of `pytest-mip`. + +- ``entrypoint.sh``: Entrypoint script of the Docker image which runs pytest with the options ``--html``, ``--self-contained-html``, ``--css``, and ``--junitxml``. Arguments given to the Docker image are passed to `pytest`. + +- ``report/``: Store HTML / CSS / JavaScript files related to the HTML report. + + * ``index.html``: Main reporting web page that contains an history of the tests (run up to the present moment) in the form of a table with links to their corresponding `pytest-html` reports. + + * ``js/reports-parser.js``: Define functions in Javascript to dynamically parse the `pytest-html` report in JUnitXML format and build the table of the reporting web page (`index.html`). + + * ``js/table-sorter.js``: Define functions in Javascript to sort by date the table of tests of the reporting web page (`index.html`). + + * ``css/style.css``: Define CSS for the main reporting web page (`index.html`). + + * ``logo_small.png``: Ebrains logo displayed at the left-hand side of the title of the reporting web page (`index.html`). + +- ``tests/``: Store all files necessary for the tests. + + * ``pytest.ini``: Define configuration options for `pytest`. + + * ``pytest_report.css``: Define CSS for pytest the HTML report generated by `pytest-html`. + + * ``conftest.py``: Define fixtures that are shared between all pytest tests, such as `selenium_driver` which connects with a selenium Google Chrome web driver, or `pytest_runtest_makereport` which is run as a post hook and allows the insertion of screenshot when a test fails in the report table. + + * ``basetest.py``: Define a base class `BaseMIPFederationTest` for the pytest tests. + + * ``test__federation.py``: Pytest tests for the MIP federation ``. + + * ``test_federation.py.template``: Pytest tests template to be used for a new federation. + +- ``.gitignore``: Specify intentionally untracked files that Git should ignore such as the file `.DS_Store` generated by Mac OSX computers or the folders `__pycache__`. + +- ``.zenodo.json``: Metadata for publication of `pytest-mip` to Zenodo, which includes among others the list of authors. + +- ``.github/workflows/build-test-report.yml``: Describe the GitHub Action workflow that builds the Docker container image, runs the tests, generates and deploys the report to GitHub Page. It consists of several "so-called" stages that are presented in the diagram below. + + ```mermaid + graph TB + A["Checkout"] ~~~|"Checkout the GitHub repository"| A + B["Set up Docker Buildx"] ~~~|"Use GitHub Action docker/setup-buildx-action
to set up Docker Buildx"| B + C[Install JQ] ~~~|"Install JQ, a package to manipulate JSON files"| C + D["Setup project parameters"] ~~~|"Set the variables UserID and UserPWD
from the project environment variables
$EBRAINSUSERID and $EBRAINSUSERPWD"| D + E[Build and push] ~~~|"Use GitHub Action docker/build-push-action
to build the Docker image"| E + F[Get current date] ~~~|"Get the current date to use
in the generation of folder and file names
to store the reports"| F + G[Run integration tests of
the up-and-running MIP federations] ~~~|"Run the tests with the built Docker image"| G + H{Success?} + I["Update Reports JSON (SUCCESS)"] + J["Update Reports JSON (FAILURE)"] ~~~|"Add item to the reports.json
describing the test date, link to its report,
and different test status"| J + K[Deploy report to GitHub Page] ~~~|"Deploy the reporting website
with updated content to
hbpmedical.github.io/pytest-mip"| K + A --> B --> C --> D --> E --> F --> G --> H + H -->|YES| I --> K + H -->|NO| J --> K + ``` + +## Installation instructions -#### Prerequisites +### Prerequisites This tool is intended to be run using its Docker image and so Docker is required to be installed. -#### How to build the Docker image +### How to build the Docker image 1. Clone the repository @@ -35,9 +100,9 @@ This tool is intended to be run using its Docker image and so Docker is required $ docker build -t pytest_mip . ``` -### How to run the tests using the Docker image +## How to run `pytest-mip` using the Docker image -#### Test all MIP federations +### Test all MIP federations Once `pytest_mip` is built, you can execute the tests for all federations as follows: @@ -49,12 +114,12 @@ Once `pytest_mip` is built, you can execute the tests for all federations as fol Once done, you can check the generated report by (1) going to the `/local/path/to/report` folder, and (2) opening the `ìndex.html` file in your favorite browser. -#### Test a specific MIP federation +### Test a specific MIP federation You can run the following command to test a specific federation: ```bash - $ docker run (-v /local/path/to/report:/app/report) -t pytest_mip test_.py + $ docker run (-v /local/path/to/report:/app/report) -t pytest-mip test_.py ``` where `` designs a specific federation. Note that a report can similarly be generated in your folder of wish (such as `/local/path/to/report`) by using the `-v /local/path/to/report:/app/report` option in the docker run command. @@ -117,66 +182,6 @@ This can be achieved by following the steps below. $ sh ../entrypoint.sh test_.py ``` -## Under the Hood - -This project contains the following files and folders: - -- ``requirements.txt``: List all the Python dependencies for the `pytest-mip` project. - -- ``Dockerfile``: Contain all the commands to assemble the Docker container image of `pytest-mip`. - -- ``entrypoint.sh``: Entrypoint script of the Docker image which runs pytest with the options ``--html``, ``--self-contained-html``, ``--css``, and ``--junitxml``. Arguments given to the Docker image are passed to `pytest`. - -- ``report/``: Store HTML / CSS / JavaScript files related to the HTML report. - - * ``index.html``: Main reporting web page that contains an history of the tests (run up to the present moment) in the form of a table with links to their corresponding `pytest-html` reports. - - * ``js/reports-parser.js``: Define functions in Javascript to dynamically parse the `pytest-html` report in JUnitXML format and build the table of the reporting web page (`index.html`). - - * ``js/table-sorter.js``: Define functions in Javascript to sort by date the table of tests of the reporting web page (`index.html`). - - * ``css/style.css``: Define CSS for the main reporting web page (`index.html`). - - * ``logo_small.png``: Ebrains logo displayed at the left-hand side of the title of the reporting web page (`index.html`). - -- ``tests/``: Store all files necessary for the tests. - - * ``pytest.ini``: Define configuration options for `pytest`. - - * ``pytest_report.css``: Define CSS for pytest the HTML report generated by `pytest-html`. - - * ``conftest.py``: Define fixtures that are shared between all pytest tests, such as `selenium_driver` which connects with a selenium Google Chrome web driver, or `pytest_runtest_makereport` which is run as a post hook and allows the insertion of screenshot when a test fails in the report table. - - * ``basetest.py``: Define a base class `BaseMIPFederationTest` for the pytest tests. - - * ``test__federation.py``: Pytest tests for the MIP federation ``. - - * ``test_federation.py.template``: Pytest tests template to be used for a new federation. - -- ``.gitignore``: Specify intentionally untracked files that Git should ignore such as the file `.DS_Store` generated by Mac OSX computers or the folders `__pycache__`. - -- ``.zenodo.json``: Metadata for publication of `pytest-mip` to Zenodo, which includes among others the list of authors. - -- ``.github/workflows/build-test-report.yml``: Describe the GitHub Action workflow that builds the Docker container image, runs the tests, generates and deploys the report to GitHub Page. It consists of several "so-called" stages that are presented in the diagram below. - - ```mermaid - graph TB - A["Checkout"] ~~~|"Checkout the GitHub repository"| A - B["Set up Docker Buildx"] ~~~|"Use GitHub Action docker/setup-buildx-action
to set up Docker Buildx"| B - C[Install JQ] ~~~|"Install JQ, a package to manipulate JSON files"| C - D["Setup project parameters"] ~~~|"Set the variables UserID and UserPWD
from the project environment variables
$EBRAINSUSERID and $EBRAINSUSERPWD"| D - E[Build and push] ~~~|"Use GitHub Action docker/build-push-action
to build the Docker image"| E - F[Get current date] ~~~|"Get the current date to use
in the generation of folder and file names
to store the reports"| F - G[Run integration tests of
the up-and-running MIP federations] ~~~|"Run the tests with the built Docker image"| G - H{Success?} - I["Update Reports JSON (SUCCESS)"] - J["Update Reports JSON (FAILURE)"] ~~~|"Add item to the reports.json
describing the test date, link to its report,
and different test status"| J - K[Deploy report to GitHub Page] ~~~|"Deploy the reporting website
with updated content to
hbpmedical.github.io/pytest-mip"| K - A --> B --> C --> D --> E --> F --> G --> H - H -->|YES| I --> K - H -->|NO| J --> K - ``` - ## Funding This project received funding from the European Union's H2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3, as part the Medical Informatics Platform (MIP)).