Human-in-the-loop for computer vision assurance: A survey

Human-in-the-loop (HITL), a key branch of Human-Computer Interaction (HCI), is increasingly proposed in the research literature as a key assurance method for automated analyses and predictive application designs. As the need increases to improve methods in Artificial Intelligence (AI) model training, optimize systems performance, provide AI explainability, and monitor AI system operations, the concept of HITL is gaining traction due to its value in solving these challenges. This survey of existing works on HITL from a computer vision system design perspective focuses on the following AI assurance principles: (1) improved data assurance, such as data preparation or automated data labeling; (2) algorithmic assurance, such as managing uncertainty and AI trustworthiness; and (3) critical limitations and capabilities introduced by HITL into a system's operational efficiency. We survey prior work within these foci, including technical strengths and weaknesses of novel approaches and ongoing research. This review of the state of the art in HITL computer vision research supports an informed discussion of considerations and future opportunities in this critical space.