Testing on our private mobile device farm using STF

0.00 avg. rating (0% score) - 0 votes

Naukri focuses on providing the best customer experience to the user irrespective of whatever platform he is using. In the field of mobile app testing, we have to make sure that testing of apps is done on maximum possible mobile devices (especially the Android mobiles having a large user base of Naukri). As there are thousands of Android devices in the market today, one of the biggest challenges we faced while testing the mobile applications is the selection and management of these devices.

While the selection is done based on the number of users we have on different screen resolutions and Android versions, the challenge arises to make these devices available to everyone in the team. As there are bugs that can only be triggered on a given hardware and software configuration, it is very important for tester to test the mobile web and native applications on every available device.

 

Problem Statement: In Naukri, we have multiple teams involved in development of native mobile apps and mobile websites. We have a range of physical mobile devices which are kept and managed by mobile app team for testing purposes. Previously, testers from different teams used to take the mobile devices from app team on requirement basis and return the same once the testing is done. Mobile team used to keep records in an excel sheet to update the names of users who were issued the physical mobile phones. This process had various loopholes such as:

  1. A team in need may not be able to get the device as some other team is using it, which may lead to a QA miss.
  2. If someone forgot to update the sheet, a lot of time was wasted in searching the device.
  3. As humans, people tend to forget returning the devices even after they have finished their testing.
  4. Gathering all the required devices(at once or one by one) for testing was tedious and time consuming as it depended on the devices being available at that time.
  5. New Joiners may not be aware of existing devices in the company to test the functionality.

 

Solution: To overcome above problems, we identified the need of a mobile cloud from where every team can access the android mobile devices remotely. This is when we came to know about Smartphone Test Farm (STF), which is an open source web application. It controls and manages real smartphone devices from our browser.

STF@Naukri for mobile testing

Implementation: We have installed STF on a Linux machine by following below steps:

  1. Run the below command to install STF and its dependencies:

$ curl -s https://gist.githubusercontent.com/MiteshShah/62cd0923b9068a927dae/raw/STF | sudo bash

STF should be installed successfully. For installation details refer: https://github.com/openstf/stf

  1. Connect multiple android mobile devices to the Linux machine using USB hub or cables.
  2. Open a terminal and run: sudo rethinkdb
  3. Open another terminal and run:

sudo stf  local –public-ip <your internal network IP here>

  1. Enter IP Address:7100 in your web browser to access the connected physical devices in your machine.

 

Challenges faced – resolved: Once we started using STF we faced certain issues related to accessibility & availability. In order to make it seamless we did the following modifications.

  1. Since STF is accessible using an IP address, the first issue we faced was that STF occasionally went down as system IP kept changing. We resolved it by getting a static IP mapped to a Lan port.
  2. To ensure its up all the time, we also scheduled a cron which runs every minute & restarts STF service if it’s down.
  3. Remembering the IP for STF was difficult for users who wanted to access the system. We then mapped the IP:Port combination to a domain name(using HAProxy)which was easy to remember. This also enables us to have capability to change the underlying IP in future, without any changes at user level.
  4. Since the system is accessed remotely, we educated users to use it over LAN connection instead of WiFi for reducing network latency.

 

Conclusion: In addition to remotely controlling mobile devices from the browser, STF provides many useful features like screenshots, device logs, drag and drop installation, copy and paste files etc from your machine. These can be referred from official STF github page.

This tool helped us significantly in following ways:

  1. Any team member can remotely control the mobile devices using his mouse and keyboard and perform real time testing.
  2. Increased availability of devices as the device is listed as available as soon as a tester finishes testing.
  3. Teams can now monitor the connected devices and the person using a particular device.
  4. Reduced overhead of managing the test devices to a large extent.
  5. Relieved us from trouble of searching ‘lost’ mobile devices and their power cords.
  6. Testers can do real time testing on every device using their browser only.

 

References:

  1. https://openstf.io/
  2. https://miteshshah.github.io/sysadmin/how-to-setup-smartphone-test-farm-stf/