HPC cloud optimisation

How The Server Labs helped Genomics England to optimise the cloud environments for their bioinformatics workloads

When Genomics England were planning a new bioinformatics pipeline solution, they wanted to ensure it ran in an optimal HPC environment. The Server Labs carried out a detailed analysis of every HPC option, so that Genomics England could be 100% sure they had the most efficient and cost effective platform.

Value at a glance

Ensuring the best HPC environment for bioinformatics pipeline
Detailed evaluation of HPC options
Expert recommendation
Enabling informed decision making
Recommending a platform for performance and scale

Value at a glance

Ensuring the best HPC environment for bioinformatics pipeline
Detailed evaluation of HPC options
Expert recommendation
Enabling informed decision making
Recommending a platform for performance and scale

Advanced Genomic Healthcare

A new bioinformatics pipeline solution

Genomics England Limited (GEL) – owned by the UK Department of Health and Social Services - is the world’s largest community in genomic healthcare and medical research. Using its genomic datasets, GEL powers lifesciences research that results in life-transforming medicines, treatments and diagnostics.

The bioinformatics pipeline is crucial to GEL’s continued ground-breaking work. GEL had been using the ‘Bertha’ suite of tools to run bioinformatics code in production, but had selected a new ‘Genie’ system, designed to:

Ingest genomic and clinical data
Orchestrate bioinformatics workflows to automate execution of patients' genome analysis
Host bioinformatics analysis code
Test new/modified workflows
Manage and monitor workflow runs

The Genie solution needed an optimal High Performance Computing (HPC) environment to run effectively and efficiently, and GEL wanted to evaluate the options before finalising their HPC set-up.

The HPC demands of the new solution

GEL wanted eliminate the issues they’d preivously had, by creating an HPC environment that could:

Run workflows at scale (bursting to AWS)
Use compute infrastructure on the cloud in addition to on-prem
Optimise workflows for cost by picking the most cost-efficient compute set-up
Support Genie’s selection of a standard workflow orchestration tool
Run workflows using different types of accelerated computing
Port workflows to different infrastructures

Additionally, they wanted to identify a Disaster Recovery setup for the existing pipelines whilst the new one is being set up.

Image of a massive data centre, with banks of servers

The HPC demands of the new solution

GEL wanted eliminate the issues they’d preivously had, by creating an HPC environment that could:

Run workflows at scale (bursting to AWS)
Use compute infrastructure on the cloud in addition to on-prem
Optimise workflows for cost by picking the most cost-efficient compute set-up
Support Genie’s selection of a standard workflow orchestration tool
Run workflows using different types of accelerated computing
Port workflows to different infrastructures

Additionally, they wanted to identify a Disaster Recovery setup for the existing pipelines whilst the new one is being set up.

Calling on the experts

The Server Labs' approach

GEL had worked with The Server Labs (TSL) in the past and knew that their cloud expertise would be invaluable in helping them select the best environment. They asked TSL to evaluate the different options so that they could make a highly informed decision.

TSL tested the workloads on different compute environments, documented the results and provided GEL with observations and recommendations. TSL set up and ran four different architecture patterns for hybrid computing for the Genie workload:
• Pattern 1 - Synced storage
• Pattern 2 - Shared storage
• Pattern 3 - Single executor
• Pattern 4 - Pattern 2 with single orchestrator

TSL trialled the following technologies:
Compute Orchestration
1. AWS Batch

C5 cpu nodes + F1.2xlarge fpga nodes
M5 cpu nodes + F1.2xlarge fpga nodes
R5 cpu nodes + F1.2xlarge fpga nodes
'Optimal' cpu nodes + F1 fpga nodes
Best performing node + F1.4xlarge fpga nodes

2. IBM LSF

Job queue with all nodes on-prem
Job queue with all nodes on AWS
Job queues with nodes on both AWS and on-prem

Fast-tier / Persistent Storage
1. AWS FSx for Lustre

Mounted on nodes on AWS
Mounted on nodes on-prem

2. Weka on-prem

Mounted on nodes on-prem
Mounted on nodes on AWS

3. Weka on AWS

Mounted on nodes on-prem
Mounted on nodes on AWS

4. S3

Staged and run on on-prem
Staged and run on AWS

8 key recommendations

TSL provided detailed, documented test outcomes: • CPU and elapsed timings for different scenarios • Costings for the different technology configurations TSL provided eight key recommendations, as well as areas for further exploration.

8

Key recommendations

Timings

CPU and elapsed timings for different scnarios

Costings

Costings for the different technology configurations

Business and technical goals achieved

Based on TSL’s feedback and recommendations, GEL now has a clear understanding of how to provide the best compute environment for different Genie workloads, and the associated costs of each.

Get in touch

Image of network of lights, with images of brains rising up from the network

Business and technical goals achieved

Based on TSL’s feedback and recommendations, GEL now has a clear understanding of how to provide the best compute environment for different Genie workloads, and the associated costs of each.

Get in touch

Selecting the right environment is essential to the performance of the bioinformatics pipeline workload. Testing every option means GEL now know they have the optimal HPC setup.