Machine Vision for Patient Monitoring in the Neonatal Intensive Care Unit

Continuous patient monitoring of newborns in the neonatal intensive care unit (NICU) is often performed with wired sensors which can be cumbersome, can interfere with parental bonding, and can irritate the patient's fragile skin. Non-contact video-based patient monitoring systems are therefore a preferrable solution. While a multitude of high-performing machine vision technologies have been successfully implemented on an adult population, such methods often fail in neonatal population. In this thesis, we assess state-of-the-art adult-based methods to bridge the gap to an understudied neonatal population in the NICU environment. To this end, several important machine vision concepts are investigated, including scene understanding, image classification, face detection, semantic segmentation, motion detection, face tracking, and heart rate estimation. In each of these areas, we assess the state-of-the-art and identify its applicability to a neonatal population. In cases where serious limitations are observed, this thesis pushes the state-of-the-art and implements new techniques more suitable for newborns. Finally, a non-contact neonatal heart rate monitoring pipeline is created using multiple research contributions in this thesis. Doing so, we obtain a vital sign decision support tool for clinical use by estimating the uncertainty in each research contributions and demonstrating how errors can propagate from one to another. Thirty-three newborns admitted to the NICU at the Children's Hospital of Eastern Ontario were recorded using a depth-sensing camera, which simultaneously captures color, depth and near-infrared videos, thereby acquiring pertinent data in all lighting conditions. Gold standard event annotations and physiologic data were recorded simultaneously as ground truth data for the development of machine vision models. Our proposed approach includes a combination of machine vision, deep learning, image processing, and signal processing techniques to overcome environmental factors such as lighting variations, occlusion, and motion artifacts. This thesis implements an efficient, robust, and reliable prototype neonatal monitoring system for potential future deployment in hospital settings. To this end, this research aimed to exploit transfer learning from state-of-the-art models to address problems such as complex NICU scenes, variations in newborn's visual features, data scarcity, and class imbalance which are often observed in clinical research and neonatal monitoring applications.