Bulk-FHIR-Validation/README.md

---
gitea: none
include_toc: true
---

# Bulk FHIR validation

Dockerized Open Source environment for **bulk FHIR validation of FHIR resources**.

## Aggregation and presentation of bulk validation results

This bulk FHIR validation environment **aggregates/groups and presents validation results of bulk FHIR validation** of FHIR Search results.

![Bulk FHIR validation](bulk-fhir-validation.png)


## Based on open standards and powerfull and flexible Open Source Software

Therefore this validation environment uses following standards and Open Source Software by the Python Library [fhirvalidation.py](home/fhirvalidation.py):

- Loading FHIR resources to be validated by [FHIR search](https://www.hl7.org/fhir/search.html) (for documentation see section "Select resources to be validated by FHIR Search parameters" below)

- [FHIR validation](https://www.hl7.org/fhir/validation.html#op) by [HAPI FHIR Validator](https://hapifhir.io/hapi-fhir/docs/validation/introduction.html) configured by Docker environment variables

- Aggregation by [Python Pandas](https://pandas.pydata.org/docs/user_guide/index.html) dataframe

- Presentation of validation results in web UI by [Jupyter Lab](https://jupyterlab.readthedocs.io/en/latest/)


## Architecture

![Software architecture](bulk-fhir-validator.drawio.png)

## Installation and Configuration

### Setup FHIR Packages

Download the FHIR NPM Packages of the [German MII Core Dataset modules](https://www.medizininformatik-initiative.de/de/uebersicht-ueber-versionen-der-kerndatensatz-module) (Kerndatensatz der Medizininformatik Initiative) to the directory `fhir-packages`.

E.g. by running [download-packages.sh](download-packages.sh):

``
bash download-packages.sh
``

If you want to use other FHIR packages, download the NPM packages to the fhir-packages directory and set them up by the environment variables of the HAPI validation service.

The environment variable names is derived from config section `implementationguides` in HAPIs [application.yaml](https://github.com/hapifhir/hapi-fhir-jpaserver-starter/blob/master/src/main/resources/application.yaml)


### Create config file

Copy .env.example to .env (so .env which will contain your credentials will be excluded from the git repo by .gitignore):

`cp .env.example .env`


### Setup your FHIR server parameters

Edit [.env](.env.example) and set up custom parameters like the URL for your FHIR Server.


### Setup initial password for Jupyter Lab UI

Set a custom initial token in variable `JUPYTER_TOKEN` in `.env`


### Start validation environment

Start the validation environment by

``
docker compose up -d
``

## Usage

### Web UI

Access the [web user interface of Jupyter Lab](https://jupyterlab.readthedocs.io/en/latest/) on the configured (default: 80) port:
http://yourserver/


#### Login

Login with the initial password / token you configured in .env


#### Start validation

Now you can start the validation and aggregation of validation results.

Therefore run the Jupyter Notebook [fhir-validation.ipynb](home/fhir-validation.ipynb).



#### Navigate validation results

You can navigate the validation results by "Table of Content" of Jupyter Lab. Therefore switch the left navigation bar from "File browser" to "Table of Contents".


#### User documentation

The further user documentation is embedded in the Jupyter Notebook:

The different outputs are described in markdown cells and used parameters are described in the code cells.


### Select resources to be validated by FHIR Search parameters

You can select/filter the resources to be validated by [FHIR search](https://www.hl7.org/fhir/search.html) parameters.

For filter options you can set `search_parameters`, see [FHIR search common parameters for all resource types](https://www.hl7.org/fhir/search.html#standard), as well as additional FHIR search parameters for certain resource types like [Patient](https://www.hl7.org/fhir/patient.html#search), [Condition](https://www.hl7.org/fhir/condition.html#search), [Observation](https://www.hl7.org/fhir/observation.html#search), ...


### Python library

If you dont want to use Jupyter Lab as a user interface (e.g. if you want to generate markdown for CI/CD reports), you can use the Python library [fhirvalidation.py](home/fhirvalidation.py) returning a [pandas](https://pandas.pydata.org/docs/user_guide/index.html) dataframe independent from Jupyter Lab.

In the Jupyter Notebook, you can find documentation on how to use the library, including example with code snippets.