CalorieCam360: Simultaneous Eating Action Recognition of Multiple People Using an Omnidirectional Camera

In recent years, as people become more health-conscious, dietary management has become increasingly important. Existing methods record only one person’s meals or eating movements, but cannot record the meals of multiple people at the same time. Therefore, we aim to record the meals of all people around a dining table using an omnidirectional camera simultaneously. In this study, we propose CalorieCam360, a system that records the entire dining table using only an omnidirectional camera and a smartphone. Note that all the processing is done inside the smartphone application without using any external servers. Since the images from the omnidirectional camera are distorted and cannot be used for detection as they are, the distortion is corrected using plane projection. The corrected images are used to detect rectangular objects that serve as references for object size, and the area is calculated by combining object detection and region segmentation to estimate the amount of calories from the area. The system then uses person detection and region segmentation to track the person and the food and records the amount of food consumed and its calorie content for each person. We demonstrate that CalorieCam360 can record an entire meal at once for multiple people around the table.