Towards the Implementation of a Mobile Application Testing Infrastructure at Von Braun Labs

With the massive adoption of mobile devices, it became more mandatory for developers to provide high-quality applications. Nowadays, mobile devices are used for different purposes: entertainment, shopping, banking, and communication. Moreover, mobile devices can communicate and exchange information with various IoT devices distributed across the city. However, mobile application testing has different challenges when compared to other types of applications (i.e., desktop and client-server applications). First, we must consider mobile devices' different characteristics and limitations, such as connectivity, screen size, density, sensors, and limited battery. Second, there is a wide range of mobile devices from diverse vendors and models. Hence, there is a need to consider different device configurations to reduce compatibility issues that may occur in a high-fragmented ecosystem. In this case, several tools and services with various features and business models aim to run tests on multiple devices. In this practical experience report, we present the initial results of implementing a testing tool/service at Von Braun Labs to support the execution of tests across multiple Android devices. The stakeholders stated the need to (i) execute the tests on physical devices; and (ii) the tool/service must support tests that interact with a specialized IoT device. We start the study by comparing different tools/services to select the most suitable one for Von Braun Labs. We propose a comparison framework to help evaluate six tools/services based on their technical, usability, and customization features. Then, we present a case study with an app from Von Braun Labs to validate the selected testing environment. Finally, we discuss the lessons learned, contributions, and future directions, pinpointing the need for a testing process since the beginning of the development project and the importance of lessening the gap between academia and industry.