Bulk-FHIR-Validation/README.md

---
gitea: none
include_toc: true
---

# Bulk FHIR validation

Dockerized Open Source environment for **bulk FHIR validation of FHIR resources**.

## Aggregation and presentation of bulk validation results

This bulk FHIR validation environment **aggregates/groups and presents validation results of bulk FHIR validation** of FHIR Search results.

![Bulk FHIR validation](bulk-fhir-validation.png)


## Based on open standards and powerfull and flexible Open Source Software

Therefore this validation environment uses following standards and Open Source Software by the Python Library [fhirvalidation.py](home/fhirvalidation.py):

- Loading FHIR resources to be validated by [FHIR search](https://www.hl7.org/fhir/search.html) (for documentation see section "Select resources to be validated by FHIR Search parameters" below)

- [FHIR validation](https://www.hl7.org/fhir/validation.html#op) by [HAPI FHIR Validator](https://hapifhir.io/hapi-fhir/docs/validation/introduction.html) configured by Docker environment variables

- Aggregation by [Python Pandas](https://pandas.pydata.org/docs/user_guide/index.html) dataframe

- Presentation of validation results in web UI by [Jupyter Lab](https://jupyterlab.readthedocs.io/en/latest/)


## Architecture

![Software architecture](bulk-fhir-validator.drawio.png)

## Installation and Configuration

### Setup FHIR Packages

Download the FHIR NPM Packages of the [German MII Core Dataset modules](https://www.medizininformatik-initiative.de/de/uebersicht-ueber-versionen-der-kerndatensatz-module) (Kerndatensatz der Medizininformatik Initiative) to the directory `packages`.

E.g. by running [download-packages.sh](download-packages.sh):

``
bash download-packages.sh
``

If you want to use other FHIR packages, download the FHIR NPM packages to the packages directory and set them up by the environment variables of the HAPI validation service.

The environment variable names is derived from config section `implementationguides` in HAPIs [application.yaml](https://github.com/hapifhir/hapi-fhir-jpaserver-starter/blob/master/src/main/resources/application.yaml)


### Create config file

Copy .env.example to .env (so .env which will contain your credentials will be excluded from the git repo by .gitignore):

`cp .env.example .env`


### Setup your FHIR server parameters

Edit [.env](.env.example) and set up custom parameters like the URL for your FHIR Server.


### Setup initial password for Jupyter Lab UI

Set a custom initial token in variable `JUPYTER_TOKEN` in `.env`


### Start validation environment

Start the validation environment by

``
docker compose up -d
``

## Usage

### Web UI

Access the [web user interface of Jupyter Lab](https://jupyterlab.readthedocs.io/en/latest/) on the configured (default: 80) port:
http://yourserver/


#### Login

Login with the initial password / token you configured in .env


#### Start validation

Now you can start the validation and aggregation of validation results.

Therefore run the Jupyter Notebook [fhir-validation.ipynb](home/fhir-validation.ipynb).



#### Navigate validation results

You can navigate the validation results by "Table of Content" of Jupyter Lab. Therefore switch the left navigation bar from "File browser" to "Table of Contents".


#### User documentation

The further user documentation is embedded in the Jupyter Notebook:

The different outputs are described in markdown cells and used parameters are described in the code cells.


### Select resources to be validated by FHIR Search parameters

You can select/filter the resources to be validated by [FHIR search](https://www.hl7.org/fhir/search.html) parameters.

For filter options you can set `search_parameters`, see [FHIR search common parameters for all resource types](https://www.hl7.org/fhir/search.html#standard), as well as additional FHIR search parameters for certain resource types like [Patient](https://www.hl7.org/fhir/patient.html#search), [Condition](https://www.hl7.org/fhir/condition.html#search), [Observation](https://www.hl7.org/fhir/observation.html#search), ...


### Python library

If you dont want to use Jupyter Lab as a user interface (e.g. if you want to generate markdown for CI/CD reports), you can use the Python library [fhirvalidation.py](home/fhirvalidation.py) returning a [pandas](https://pandas.pydata.org/docs/user_guide/index.html) dataframe independent from Jupyter Lab.

In the Jupyter Notebook, you can find documentation on how to use the library, including example with code snippets.