Analyzing 3D Volume Segmentation by Low-level Perceptual Cues, High-level Cognitive Tasks, and Decision-making Processes

3D volume segmentation is a fundamental task in many scientific and medical applications. Producing accurate segmentations efficiently is challenging, in part due to low imaging data quality (e.g., noise and low image resolution) and ambiguity in the data that can only be resolved with higher-level knowledge of the structure. Automatic algorithms do exist, but there are many use cases where they fail. The gold standard is still manual segmentation or review. Unfortunately, even for an expert, manual segmentation is laborious, time consuming, and prone to errors. Existing 3D segmentation tools are often designed based on the underlying algorithm, and do not take into account human mental models, their lower-level perception abilities, and higher-level cognitive tasks. Our goal is to analyze manual segmentation using the critical decision method (CDM) in order to gain a better understanding of the low-level (perceptual and marking) actions and higher-level decision-making processes that segmenters use. A key challenge we faced is that decision-making consists of an accumulated set of low-level visual-spatial decisions that are inter-related and difficult to articulate verbally. To address this, we developed a novel hybrid protocol which integrates CDM with eye-tracking, observation, and targeted questions. In this paper, we develop and validate data coding schemes for this hybrid data set that discern segmenters' low-level actions, higher-level cognitive tasks, overall task structures, and decision-making processes. We successfully detect the visual processing changes based on tasks sequences and micro decisions reflected in the eye-gaze data and identified different segmentation decision strategies utilized by the segmenters.