Leo Joskowicz, PhD (Hebrew University of Jerusalem, Israel) – Quantifying the observer variability in volumetric structure segmentations: a large-scale study and a method
Segmentation of anatomical structures and pathologies in medical images is a fundamental technical problem in medical image processing. Producing accurate and reliable segmentations for clinical use is expensive, time consuming, and requires technical expertise. Often times, it is unclear what is the accuracy and quality of these segmentations because there is no reference ground truth to compare them to. In this talk we present: 1) a new framework for segmentation variability quantification based on segmentation priors and sensitivity analysis; 2) a method for estimating segmentation variability with no ground truth, and 3) a large-scale manual delineation study to quantify the actual segmentation variability in which 11 radiologists manually delineated the contours of liver tumors, lung tumors, kidneys, and brain hematomas in 3,193 CT slices from 18 representative CT scans. Our results show that segmentation variability spans a wide range depending on the structure of interest and that it can be accurately estimated independently of the segmentation method used and with no ground truth.
Joint work with: D. Cohen, Dr. N. Caplan and Prof. J. Sosna, Hadassah University Medical Center
Leo Joskowicz is a Professor at the School of Engineering and Computer Science at the Hebrew University of Jerusalem, Israel, where he conducts research in computer-assisted surgery, computer-aided mechanical design, computational geometry, and robotics since 1995. He obtained his PhD in Computer Science at the Courant Institute of Mathematical Sciences, New York University, in 1988 and was a Research Scientist was at the IBM T.J. Watson Research Center, Yorktown Heights, New York, USA. where he conducted research in intelligent computer-aided design and computer-aided orthopaedic surgery. From 2001 to 2009 he was the Director of the Leibniz Center for Research in Computer Science.
Tal Arbel, PhD (McGill University, Canada) –
Challenging Conventional Segmentation Evaluation Metrics in the context of Focal Pathology (e.g. lesion) Segmentation from Patient Images
In the context of automatic segmentation for medical images, overlap and boundary distance metrics (e.g. Hausdorff distance, DICE coefficients) define the standard for quantifying the performance an algorithm against structural delineations by experts. These metrics, adopted from computer vision, are well-suited to the context of healthy structure segmentation or segmentation of a single pathological structure, as they typically adhere to the underlying assumptions that: 1) the structure in question exists and makes up a substantial portion of the region of interest, and 2) variability in both ground truth and in automatically generated delineations mainly consist of differences in voxel assignments.
In this talk, we will illustrate a number of challenges in applying these segmentation evaluation metrics to delineating multiple pathological structures (e.g. lesions, tumours) in patient images, where clinical objectives can be substantially different, and associated assumptions violated. We focus on the illustrative context of Multiple Sclerosis lesion segmentation, where challenges include, but are not limited to: 1) inter/intra-patient lesion variability in terms of size (spanning from a few voxels to over one hundred), count, position and shape; 2) inter-rater variability including discrepancies regarding lesion existence; 3) clinical objectives which require detection and segmentation of *all* lesions in order to estimate treatment efficacy (in clinical trials and in the clinic). We provide illustrative examples of challenges and requirements placed on the main industrial clinical trial analysis system used in the development of the majority of new MS treatments currently available worldwide, as well as the process and resulting lesion labels provided by their trained neuro-radiologists.
Joint work with Dr. Arnold, neurologist at the Montreal Neurological Institute and President of NeuroRx Research.
Tal Arbel is a professor in the Department of Electrical & Computer Engineering and Director of the Probabilistic Vision Group and Medical Imaging Lab in the Centre for Intelligent Machines at McGill University, Montreal, Canada. Her research focuses on the development of probabilistic and machine learning methods in computer vision for medical image analysis, for a wide range of applications in neurology and neurosurgery. She has particular extensive expertise in developing probabilistic graphical models for brain tumour/lesion detection and segmentation. Recent work is focused on modeling uncertainty in deep learning networks for medical image analysis and on developing machine learning methods for the automatic identification of biomarkers predictive of future neurodegenerative disease progression. She has co-organized a number of major international conferences, including serving as co-organizer and satellite events chair for MICCAI 2017, and area chair/program committee member for CVPR and MICCAI. She is currently an Associate Editor for IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) and the Journal of Computer Vision and Image Understanding (CVIU).
Previously we had announced a talk by Dr. Elizabeth Krupinski, which unfortunately had to be cancelled due to scheduling problems.