Use of Workflows in Managing Experimental Data

Systematic capture and storage of experimental data is critical in any research activity. Even today, experimental data are laboriously recorded by hand in lab notebooks. Sophisticated e-lab notebooks, and subsequently, laboratory information management systems (LIMS), have evolved over the past 10 years, but are not wholeheartedly embraced by the research community due to a variety of reasons, price of commercial software packages and resistance to change being the main ones. Due to the persistence of old techniques, the recorded data are either not machine readable (if entered in a notebook), or the data provenance is not properly recorded. This significantly jeopardizes the long term use of the data, for either reproducibility or verification, as well as the ability to share it among peers or do data mining at a later date. More systematic and affordable techniques are needed to capture all the information about experimental data in a semantically rich format.

An interactive data entry and retrieval system was developed at Purdue for managing the information generated during the dry powder blending experiments  performed at Rutgers. The key component of the system is the workflow for the associated experiment. A workflow captures the operational aspect of a work procedure: how tasks are structured, who performs them, what their relative order is, how they are synchronized, how information flows to support the tasks and how tasks are being tracked. A workflow model provided the reference points for defining the data to be recorded, and designing the data entry forms. Each recorded experiment is an instance of its associated workflow. All the instances of a workflow provide the handles for all data retrieval and processing functions.

The experimental data is stored on the pharmaHUB server hosted at Purdue in normalized tables in a MySQL database, and can be retrieved using a collection of views. A graphical user interface provides a concise, pictorial representation of the underlying standard procedure, with data entry done through icon specific forms. In this paper, we will discuss the details of the system and its use at Rutgers.