Data Catalogue (DC) is a component of the Medical Informatics Platform (MIP) for the Human Brain Project. Initially, it was designed and tasked to be the single source of truth of metadata descriptions for data residing in hospitals that have joined the MIP. In the course of things, additional needs have been recognized; therefore DC ended up supporting the following:
- Presentation of the Medical Conditions to which data refer in the MIP.
- Presentation and visualisation of the Common Data Elements (CDEs) data models for the Medical Conditions in the MIP.
- Presentation and visualisation of the hospital local data models that have been mapped and harmonized to the CDE data models.
- Management (create, edit, delete) of the above (global and local) data models with version control by authorized accounts. Data models are defined via DC's GUI or are imported in XLSX format.
- Generation of metadata files in JSON format according to MIP's specifications.
The user of DC is the researcher who, prior to executing experiments with the MIP, needs to investigate the included Medical Conditions along with their semantics as well as what type of information is available in each hospital.
Another user is the data provider / manager of a hospital that has joined the MIP. The data provider / manager has to define and manage the metadata of her hospital data.
Lastly, the MIP portal itself is a DC user as it uses its REST API to get the latest CDEs versions.
The user can perform a set of actions without being logged in.
Common Data Elements (CDEs):
- View all medical conditions here
- Select the current or any previous version of the CDEs meta-data of a specific medical condition here
- Search CDEs meta-data based on variables' category or code for a medical condition version here
- View details about the meta-data here
- Download the JSON that contains the meta-data that will be used in the MIP Federated Node for a specific medical condition here
- View the hierarchy of the meta-data in an indexable tree structure here
Hospital Meta-Data:
- View the meta-data of all hospitals combined here
- View the versions of the meta-data of each hospital and search with category and/or code here
- Download the JSON that contains the meta-data of the hospital local data model here
- View the hierarchy of the meta-data in an indexable tree structure here
- View / Download / Index the results of the Quality Control Tool for the meta-data here
- View the graphical representation of how variables are mapped to CDEs and using which rule here
Data Catalogue uses a keycloak instance setup for the MIP in order to authorize users. There are 3 role types available
- administrator - has access to everything
- pathology owner - has access to everything that is included in a pathology (cde_version,hospital,local_version)
- hospital owner - has access to everything that is included in a hospital (hospital,local version)
The extra actions that the user can do with appropriate role are the following:
- Create a new medical condition (administrator/pathology owner)
- Create/delete hospital (administrator/pathology owner/hospital owner)
- Create/edit/delete a meta-data for global model using the GUI or uploading a file (administrator/pathology owner)
- Create/edit/delete a meta-data for local hospitaldata model version using the GUI or uploading a file (administrator/pathology owner/hospital owner) step1, step2, step3 step1, step2
- Download a template file that contains the schema of the meta-data that should be completed (anyone logged in) step1, step2
Important Note: Currently, Data Catalogue does not support user logging out. So, in case the user confronts any log in issues, it is advised to close the web browser entirely and try to log in again, or open a private web session and try to log in from there.
Required installed packages:
- Java 1.8 (Oracle)
- PostgreSQL 9.5
- Spring Boot 2.1
- Angular 6.2
- Angular CLI 6.2
- Typescript 2.9
- D3 5.7
Clone the repo:
git clone https://github.com/aueb-wim/DataCatalogue.gitAfter loging in PostgreSQL shell we create an empty database named datacatalog:
create database datacatalog;Install Maven. In a terminal:
sudo apt install mavenFill in the credentials that we use to login to PostgreSQL and the IP / port of the server at application.properties file in DataCatalogue:
- spring.datasource.username= yourUser
- spring.datasource.password= yourPassword
- server.port=8086
- server.address=172.16.10.138
Change the URL of the Authentication Entry Point in service/MIPSecurity.java line 150 if necessary. Run spring boot:
mvn spring-boot:runThis will initiate the back end services.
Need to have Angular and Angular CLI installed. Having Angular already installed, inside the frontend directory run:
npm installTo configure frontend, change the following files with the correct URLs:
- frontend/proxy.conf.js
- frontend/proxy.conf.json
- frontend/src/app/shared/hospital.service.ts
- frontend/src/app/components/form-upload.component.ts
- frontend/src/app/components/form-upload-cdes.component.ts
To initiate the frontend services, we use angular CLI:
cd DataCatalogue/frontend
ng serve --host 0.0.0.0Open the DataCatalogue project with any IDE (Intelij recommended). To run Spring boot, execute the class DataCatalogueSpringBootApplication. Frontend is initiated as said with ng serve.
- Admir Demiraj - AUEB/RC Team
This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.
This project/research received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Framework Partnership Agreement No. 650003 (HBP FPA) as part of HBP SGA2 T8.5.2, and under the Specific Grant Agreement No. 945539 (Human Brain Project SGA3).
Special thanks to:
- Prof. Vasilis Vassalos - Athens University of Economics and Business