Every day, scientists around the world are exploring innovative treatments for numerous diseases. Their research relies on large and increasingly complex datasets, which hold significant potential for advancing medicine and healthcare. Given the highly confidential nature of the personal data they handle, researchers must take special precautions to ensure ethical use and compliance with privacy laws. The challenge is: how can these data provide maximum benefits while minimising the risks for patients and data providers?

The SPHN (Swiss Personalised Health Network) initiative, launched in 2017, aims to make patient data from Switzerland’s five university hospitals available for national research. These data must be standardised and structured to ensure interpretability. Hospitals transfer data from their IT systems into a data platform (a data warehouse or data lake) to enable secondary use, such as for research. BioMed IT, a network for exchanging health data for research purposes in Switzerland, provides researchers with a secure environment for processing sensitive data.

BioMedIT in Detail

The BioMedIT network is based on modern and secure cloud environments at the University of Basel, ETH Zurich, and the University of Lausanne. Data providers such as hospitals and research organisations can encrypt and transfer data to the platform using these nodes. Projects from research consortia or individual researchers that wish to use sensitive data are set up as research projects in a project-specific BioMedIT environment, known as B-Space, at one of the nodes. Researchers securely access the system via two-factor authentication and extract only non-sensitive or aggregated research results while sensitive data remains on the platform.

The Swiss Institute of Bioinformatics (SIB) is responsible for the technical implementation of this initiative. BioMedIT benefits from SIB’s technical expertise, built on decades of experience in bioinformatics infrastructures.

The Journey Begins

In 2020, initial versions of the desktop application for data encryption, transfer, and decryption (sett: Secure Encryption and Transfer Tool) and the web-based user and project management solution (BioMedIT Portal) were already available. However, like the entire project, these were at a very early stage. To expand and implement the BioMedIT Portal, SIB enlisted the expertise of Karakun. Katrin Crameri, Director of Personalised Health Informatics at SIB, explains: “At that time, we found a competent and highly appreciated facilitator in Christian Ribeaud, who actively supported us in further developing the core BioMedIT tools. He was so effective that we did not want to let him go before the end of the project, contrary to our initial plans.”

Introduction of Scrum as a Foundation

At that time, the COVID-19 pandemic posed a significant challenge. The Swiss government implemented measures to protect the population, and another lockdown was imminent. These were unfavourable conditions for a collaborative software project. However, sufficient personal resources were available, as each of the three nodes of the BioMedIT network provided a full-time software developer. Better coordination was needed, which the team achieved by introducing Scrum, laying the agile foundation for the continuous development of the entire portal.

Agile Release Management

Agile release management is particularly suitable for smaller teams handling large projects. With the switch to agile software development, release versioning was reorganised in GitLab repositories. This ensured tracking of all changes and prevented work from being lost or accidentally overwritten. Code changes could not only be monitored but also rolled back if necessary. The main applications are now available as open-source software on GitLab.

Planned Extensions

The sett desktop application had to meet the highest security requirements. In addition to asymmetric encryption and signing, it needed powerful robust compression due to potential uploads of several hundred gigabytes. To make sett as user-friendly as possible, it needed to run on different operating systems and offer both a graphical user interface for data senders and a command-line interface for data analysts.

Rethinking Security

The team could not meet all these requirements using Python, which was the originally proposed implementation language. They therefore reimplemented the sett application in Rust, a programming language known for its security. They also integrated Sequoia-PGP, a Rust-based OpenPGP implementation, for encryption. This guarantees that patient data remains completely inaccessible to unauthorised parties during transfer. Finally, by introducing a standardised protocol, S3, for data transfer, performance improved significantly compared with the previously used SFTP.

Further Highlights

The Karakun-led team not only developed and enhanced the two main applications, but created an entire ecosystem, including:

• Two additional applications based on the BioMedIT Portal
• A monitoring system based on Prometheus
• Visualisation of metrics using Grafana
• Alerting and log collection based on OpenSearch

Successful Project with Great Benefit for Science

After approximately four years, developments will be finalised in 2024, marking the start of the operational phase. Thanks to all the technical innovations and extensions, nearly 1’000 end-users are now registered in the BioMedIT network. University hospitals transfer several hundred data packages through the system each year. Katrin Crameri is enthusiastic: “We have managed to provide practical and targeted solutions for mobilising and administering sensitive data in multi-centre research projects, in strict compliance with ethical and legal data protection and security requirements at all functional levels. The support from Karakun in developing the tools and processes has been highly appreciated by everyone involved in the network.”

Patients in Switzerland now receive better and more targeted treatments through the work of researchers via the BioMedIT network. We are delighted to have contributed to this exciting project!