Skin tactile surface restoration using deep learning from a mobile image: An application for virtual skincare

Background For virtual skincare using a touch feedback interface, reconstructing a 3D skin tactile surface from a mobile skin image is imperative for a dermatologist to palpate the skin surface that presents tactile characteristics of the subcutaneous tissues. However, the precise tactile reconstruction from a single view image is a challenging research problem due to varying illumination conditions. Methods In this study, a deep learning-based tactile reconstruction scheme is proposed to restore tactile properties from light distortion and reconstruct the 3D tactile surface from a mobile skin image. Our method consists of light distortion removal using deep learning, cGAN, and 3D tactile surface generation based on image gradients. Results The proposed method was tested by conducting two evaluation experiments in terms of removing light distortion and reconstructing 3D skin tactile surface in comparison with other well-known methods. The results demonstrated that our method outperforms existing other methods in both illumination-free image restoration and 3D surface reconstruction. Conclusion The proposed method is a promising approach in that tactile property distorted by illuminations can be completely restored using deep learning with a smaller training set and the precise reconstruction of 3D skin tactile surface can be achieved to be ready for a remotely touchable interface for virtual skincare applications.