Skip to content

Acceptance Test Specification

Jump to bottom
Pascal Dihé edited this page Jul 13, 2020 · 6 revisions

Acceptance Test Specification

This is the specification of the acceptance tests to be carried out by the testing team.

Reason

Before we can go live with the system, we have to perform scientific validation and quality assurance of the screening results as well as validation (acceptance tests) by users.

Goals

The main aim of these tests is to ensure that our "minimal viable product" is "fit for purpose", that is, it delivers the proposed value (European-level climate change impact screening service). Thereby, we want to eliminating usability issues (difficult to understand, too complicated, illogical or incoherent behaviour) and bugs (something is not working as expected or not working at all). While we want to collect also feedback for improving the system, our current focus is on making existing functionality work as expected rather than collecting many new feature requests for new functions.

CSIS Screening Value Proposition

This is what the CSIS offers in terms of public and free screening. Basically this is also what has to be tested:

The user can perform an "automated screening study" for a number of supported metropolitan regions in Europe to assess the impact of heat wave events on population (mortality rate) referring to a specific RCP, time period and event frequency. The user can show HC (heat wave, pluvial flood), HC-LE ("downscaled" TMRT taking local land use information from e.g. Urban Atlas into account), exposure (population distribution) as well as calculated impact (mortality rate) on a map. Simple tabular visualisation of HC, EE and IA data and CSV and GeoJSON export is available. The user can add a few "twins" (hazard twins, solution offers, showcases) and generic Adaptation Options to a printable report (PDF), simple "scenario analysis" for one indicator (mortality rate) is available, too.

Giving Feedback

We create separate issues for each organisation in the Technical Validation Repository. If the testers encounter any problem a new comment has to be added to the respective issue. This comment has to include the following information:

  • URL, e. g. https://csis.myclimateservice.eu/study/1/step/83/view/summary
  • Date & Time, e.g. 2020-02-24 08:04:00
  • Browser Version and Operating System, e.g. Firefox 73.0.1 (64-Bit) / Windows 10
  • Screenshot

Additionally:

  • Description of the expected behaviour, e.g. "the table should show some values"
  • Description of the actual behaviour, e.g. "the table shows an error message: 'not logged in' instead"
  • Description of the steps that can be taken to reproduce the issue, e.g. "select the study scenario 'XYZ' from the list"

Optional but helpful: In case of unexpected or erratic behaviour in embedded web applications like Map Component, Table Component and Scenario Analysis Tool, please press CTRL+SHIFT+i to view the Browser console.

In Firefox and Chrome there is the possibility to save the content of the console window to a text file:

image

image

This file can be added to the respective issue (drag & drop).

'Walkthrough'

Those are the actions that have to be undertaken by testers. At the end of the process, you should be able to download a PDF with the result of the personalised screening.

The description of the steps is intentionally a bit vague, since the system should be self-explaining. Fortunately, end users don't have to figure out by themselves how to use it. There are some videos available that explain the basic workflow for creating a screening study:

If you still get stuck at some point in the walkthrough please add new comment and describe the usability problem.

Starting point is https://csis.myclimateservice.eu/

  1. Login with your existing credentials or create a new user account. Hint: If you've created a new account on profile.myclimateservices.eu and you didn't receive the confirmation email with the account activation link, check your spam folder.

  2. Create a new Study of type Advanced Screening: Urban Infrastructure

  3. Add another team member from you organisation with role Team to your study. (we've asked to appoint two users for testing the system from each organisation). Hint: Of course, the 2nd team member needs a profile on myclimateservice.eu, too.

  4. Edit the study context, fill in some information about your study (study goal, sector, ...) and select a city of your choice. Hint: We are currently re-calculating input data for screening, so at the moment there are just a few cities available for screening.

  5. While still editing the study context, create at least two different study scenarios, e.g. for comparing historical climate events to future climate events.

  6. Select a Study Area on the Study Area Map.

  7. Once the new Study Area has been defined and saved, add it to the report. (include in report) Hint: The map can be zoomed and panned, different map screen shots can be added to the report.

  8. Select a data package. Hint: The study is now ready to be calculated, press the respective button to start the calculation. Hazard Local Effects and Impact Calculation will take some time. The system will keep you updated on the status of the calculation.

  9. When the calculation has finished, visit the steps HC, HC-LE and HC-EE and respective sub (data, table maps, ...), switch between different scenarios (table and maps), select / deselect some map layers, change the table sort order, export some table data, change the map view (zoom & pan), etc. and create several screen shots (include in the report) and add some titles and descriptions to the screen shots pictures (e.g. information of the selected scenario).

  10. In the RA/IA step, perform the same actions as above. Additionally, on the scenario analysis tab, change the selection of indicators and create screen shots for inclusion in report.

  11. Download and verify PDF report.

Clone this wiki locally