DolphinNext: A Graphical User Interface for Distributed Data Processing of High Throughput Genomics

Abstract

Emergence of new biomedical technologies, like next-generation sequencing (NGS) which is producing vast amounts of genomic data every day, is driving a big data revolution in biology. The dramatic increase in the volume, as well as the production rate of genomic data, has now made the data analysis new bottleneck for scientific discovery. Naturally, the need for highly-parallel data processing frameworks is greater than ever. It is also important for these frameworks to have certain design characteristics such as flexibility, portability, and reproducibility. Processing of sequencing data usually involves many different programs, each of which performs a specific step in the overall pipeline. Flexibility ensures that the pipelines can support a variety of use cases or data types without the need to modify existing pipelines or create new ones. Portability gives user the freedom to choose computational resources as he/she deems fit. Reproducibility across computing environments, which warrants credibility of the results, is a particularly important feature in the face of the sheer volume of data and complexity of the pipelines. There exist several platforms that offer graphical user interfaces for designing and execution of complex pipelines (e.g. Galaxy, GenePattern, GeneProf). Unfortunately, none of these platforms supports parallelism or portability across computing environments. To address these and additional shortcomings discussed in this paper, we have created DolphinNext, an easy-to-use graphical user interface for creating and deploying complex workflows for parallel processing of high throughput genomic data. DolphinNext relies on Nextflow which is a framework enabling scalable and reproducible workflows using software containers. The central idea behind the creation of DolphinNext is to facilitate building and deployment of complex pipelines using a graphically-enabled modular approach.