How To Grow Your Mobile Application Testing Skills
By Maxim Zheleznyy
These little mobile devices in our hands seem simple to use, compact, convenient, and extremely fast. Not everyone knows that to achieve such results, testers need to apply a huge amount of effort, a multitude of skills, and pro tips that might not be applicable when testing desktops or web applications.
Whether you are currently testing a mobile app, or want to learn more about mobile testing and what it involves; the following suggestions and ideas can help you get started and aid you with your testing journey in the mobile space.
The life of the tester is diverse and multifaceted. In order to not drown in the flow of information and to be the most effective, there are many techniques and tools a tester can learn and use to improve their testing strategy. Below we will look at different types of tools that can enhance your mobile testing.
Emulator & Simulator Tools
Emulator and simulator tools are the best friends of a tester with a small farm of devices. If you find a bug that appears only on some exotic kind of resolution (see Chinese smartphones), then you can use some of the tools listed.
XCode’s simulator is the easiest and, in most cases, the only way to simulate an iOS device with a variety of new features (like iPhone 10 X). The only limitation is that you need to have macOS compatible device such as iMac, MacBook and Mac mini.
If you are looking for a way to simulate Android devices, you should try AVD (which stands for Android Virtual Device). It’s embedded inside Android studio, and unlike XCode, can be installed on Windows and Linux.
Genymotion is my favourite Android emulation tool. With it, you can easily create and manage hundreds of different emulated Android devices.
It is important to note that simulators and emulators cannot replace real devices. Do not consider successful tests on simulators to be the same as successful tests on real devices.
Performance & Monitoring Tools
It’s very important to not only test the app but also have the ability to manage and control all actions on a device. The easiest way to control everything is to manage running an app via Android Studio and XCode, more precisely with the help of their embedded tools:
- Android monitor gives you access to such data as log messages, data analysis tools, and performance monitor.
- Instruments for iOS, in most cases, has the same features as Android monitor, but you probably should try it yourself to know for sure.
Most modern applications are tightly tied to work with the network, so it will be quite reasonable to have many different and advanced tools to manipulate network configuration.
For iOS, you could use Network Link Conditioner which is provided by Apple and is an extremely simple tool for setting up the required connection. Do you need an imitation of a bad connection on a real device? Would it be desirable to test how the application will behave with lost data packets? It’s no problem for this tool to help with these kinds of tests.
For Android, everything is somewhat more complicated. On the device itself, you can not create mock network conditions, so you'll either need to use a device emulator with a special setting or use the Network Link Conditioner (see the point above) on a real iOS device and then enable a hotspot on this device. You can then connect your Android device to the iOS hotspot. It’s an ugly looking method, but it is guaranteed to check the behaviour of a real android device without foil.
Fiddler and Charles are both are proxy tools and are beautiful in their own way. If you need to look at the application data sent/received through the network; these tools can manage those interactions. They will perform the function of the man-in-the-middle, which will allow you to view the entire contents of the data packets.
API testing tools are absolutely indispensable in case you need to test the API layer of a mobile app. Products such as Paw, Insomnia, SoapUI, and my personal favourite Postman, are some great tools to test API. Since many mobile applications are large program layers forming requests to the server and processing the responses received, it would be reasonable for us, as testers, to check what the server sends from its API methods. To get started, all you need for each of them are examples of different requests to the server (Get, Put, Post, Delete) and the responses to be received.
By using automation you can achieve more stability and increase your test coverage. The following are several tools that can help you automate your mobile applications.
Appium is the most popular current framework with a great history. It’s worth it to say that if you have experience with Selenium, it will be easy to start using Appium. If automation is a new sphere for you, try Appium. In particular, Appium desktop has a great UI to explore your app, this helped me with my first steps into mobile automation like nothing else.
Android’s Espresso and iOS’s XCUITest are other good options instead of Appium. These two tools were developed to serve a specific OS, so you can’t write one code and use it on any device. Both of these applications have a great feature which can record and repeat your interactions with an app. Appium also has this feature, but in most cases, it is not as stable and easy to use. The main issue with Espresso and XCUITest is that they are UI level test tools which means that they will perform extremely slow and require a lot of energy to maintain.
Another powerful and even faster tool for automation is Calabash. In most cases it works like Appium, you can write your tests in Ruby, unlike with Espresso and XCUITest. However, Calabash developers recently said that they will stop improving the framework. It’s a good tool, but be aware that it will soon be unsupported.
Also, UI Auto Monkey for both iOS and Android is fairly easy to start. This tool is a simple script that emulates a user pressing lots of buttons and loads all the screens at once causing chaos until your app eventually crashes. However, it is worth noting that Auto Monkey gives an extremely small benefit, and it is impossible to catch some fundamental crashes with it.
Delivery And Analysis
Delivering applications to test devices is a very important part of testing because you can’t check the latest up-to-date build until it gets to you. Various services come to the rescue and solve this problem. Below I will share the services I’ve managed to try, and also talk about the ideal place in the development process for each of them.
TestFlight and Beta testing mimic the play store and are excellent tools for testing the installation of new builds when you try to update from the current live version. These distribution tools require certificates for the app and additional time to release, so it’s pointless to use them in the early stages of the project.
HockeyApp, AppBlade, Appaloosa, TestFairy are convenient tools for distributing builds for both iOS and Android. Each of them supports the ability to download previous builds, and some even allow you to collect analytics about the use and crashes. These tools can be used both at the earlier stages of development and before the release. Many enterprise customers choose to distribute their applications through these services.
Google Drive, Dropbox, Box, and other cloud storage providers can replace the services listed earlier in Beta Testing and Alpha Testing. Version control and managing will fall on your shoulders if you decide to proceed in this direction. If you are crazy organized, then you can try to build all the processes on cloud drives. On my team, we use these services when we needed to make a backup copy of a certain version of the application, or during the very early stages of development.
Equally important is the analysis of applications because a person who is warned is armed. That's why before the release of the product, and even better, before the development, it is necessary to determine the services that will best allow you to keep your hand on the pulse of your application. Here we’ll look at two methods of analysing your application.
CrashLytics, Bugsee, AppSee, and FireBase give a very detailed analysis on the use of your applications. They almost instantly report crashes, and are full of different graphs and diagrams. Each of them have their own unique features. For example, Bugsee allows the tester to create a ticket with an error record directly from the mobile application, and FireBase was originally created as a means of quickly deploying the backend. Given all these features, it’s a good idea to investigate each tool and choose the most suitable tool for you in your current context.
Reviews in the App Store and Play Market are important to watch. Unlike our colleagues from the web world, we can always look at the real opinions of our users. Will you read the positive reviews as much as negative ones? Hardly. Will negative feedback always be useful? I doubt it. Will the time spent reading reviews be worth it? Yes! If you want to make a really high-quality product that is interesting to users, reading the reviews is important. However, it is worth pointing out that if the whole team does not work with reviews, then any insight gained will be limited. Reviewing them as a team can generate ideas and initiatives which lead to a better quality app. Cultivate and instil a love of reviews and work with them. This practice will pay off in the long run.
You could also implement feedback forms for each application so the user can contact your team without spoiling the rating in the application store. For the user, it can be easier to navigate the history of email correspondence rather than a faceless store.
Hints, Tips And Other Stuff
In any kind of development, bottlenecks can form, especially around defects generated by an app. Listed below are some working suggestions that can simplify the search for defects.
Heuristics And Mind Maps
It can be useful to use one of these mind maps or heuristic mnemonics such as “I SLICED UP FUN,” “LONG FUN CUP,” “Blink testing,” or other examples which can help you with your current testing. They can help you recognize something important about your application. If any of the heuristics or mind maps don’t seem to fit a particular project, you can always modify them or create your own heuristic or mind map.
If you have to test 4 different input fields and different combinations of each of them, then the PairWise method can be useful. With this mathematical tool, you can eliminate duplicate testing efforts and streamline your testing process.
Third Party Tools
Be sure to arm yourself with tools for logging, screen recording, and screenshots. There are suitable options with OS-included tools or programs from third-party developers.
Always check the application in portrait and landscape modes. Sometimes developers forget to disable the support of landscape mode or skip the layout of any of the necessary elements. It's frustrating and disappointing to find out your favourite program crashes at the slightest turn of the screen.
Be sure to look at how the application behaves when a bunch of other apps are running. Study what happens after the device goes to sleep, or the home button is pressed, and then returns to the active mode. See what happens when the device and application abruptly exit the active mode, like when a user receives a phone call. Below you’ll find ways to mock “under pressure” situations for the device and its apps.
Do Not Keep Activities or (DNKA)
Be sure to test your application with and without DNKA, which greatly reduces the time spent checking operations with activities. Hint: when you create a ticket for bugs that were found using DNKA, be sure to indicate that you found the defects using this tool. It will be easier for the developer to reproduce your error.
Low Battery & Low Disc Checks
Don’t forget to check your app performance with low battery and low disk space. Unfortunately, Apple doesn’t give access to any software tool which can do that, but you can test it manually. It is easier to test these things with an Android OS. The only thing you need is a simulator or emulator with a low memory setting. To mock low battery status for Android, perform the steps in this blog to get started.
OS Onboard Services
A huge number of applications use the services of the OS. These include the camera, contacts, the microphone, and so on. On an Android, the user gives access to these services when the application is downloaded, and for iOS, as well as Android 6.0 and above, permissions are given after the program is launched. Checking when permission is given for the use of onboard services is an important part of mobile testing. Requesting access to a service that isn’t used by the application is strange. Make sure your application is requesting the right access and the user is granting it before the application uses an onboard service.
When dealing with notifications, you should remember that they can be both local and server side. In addition, notifications, ideally, should lead to the screen referred to in the notification. Unfortunately, it is very common practice when clicking on the notification like: "Hey, you received a message from John" - that you end up at the main screen of the application, and not where the much-desired message from John is located. Be kind to your users, test your interactions and send them to the right screen.
There are special tools for tests related to the network, but I want to point out again that checking how an application reacts to different types of communication and transitions from one state to another is extremely important. Most users of mobile devices do not sit in a cosy office next to a WiFi router. Keep in mind where your user is actually using the application and test those places in the field, like walking down the street or taking the subway.
It is extremely important to see how the application behaves if you quickly switch between different screens (remember the monkey test mentioned earlier). In my Android user experience, an extremely large number of errors are due to the fact that with a quick transition between different screens, the necessary fields do not have time to fill with data and, subsequently, lead to a crash.
Test changing the language and time on the device. Never count on the fact that users will be synchronized with the network and automatically go to the correct time zone. If your application somehow has to change with time, for example, delivery of food after 30 minutes, then you have an endless scope for activity.
Always communicate with managers and developers. A few minutes of informal communication about the new build will provide valuable insight into their expectations, and simplify your life. Also, before testing a new build, I recommend asking developers a few questions about the build. Ask them what they have already checked and if they are aware of any risks. This information should be treated as valuable, it will help you design tests and find bugs.
Sources Of Information
A tester should be a small specialist in every single area, like a penknife. To be better informed you should obtain certain sources of information about what you are testing and how it works. Below are some useful sources of information on iOS, Android, and general mobile development; including some great blogs, guides and events to attend.
Refer to Apple’s page of strict app publication guidelines and app design guidelines, which Apple calls “Human interface guidelines”. These details are the kind to be aware of and keep track of for future releases.
Another recommended source is iOS goodies. It contains a weekly digest where you can find something interesting to read about iOS community developments. Here are a few more recommendations that could help you with your iOS testing:
- NSHipster is a great source of tips and tricks. There you will find a lot of information about development. The only negative side is that NSHipster is almost outdated, so you will probably not find any new information here.
- Timac is similar to NSHipster, but I have found this blog, more clear, with easy-to-use examples and useful pictures.
- Thomas Hanning is another worthwhile blog. He doesn’t update his feed often, but when he does, it should be considered required reading.
- Ray Wenderlich is a very valuable source of information, any iOS question or new function you may have is bound to be covered here.
- 9to5mac is another great resource. It has tons of short news pieces and reviews which you could quickly read during your morning commute or breakfast.
Google’s publication guidelines and design guidelines are good places to start for Android development and testing. You’ll find that the publication rules are pretty similar to Apple’s and might give you a deja vu feeling. Here are a few other Android sources to look through:
- Riggaroo is updated around once every three weeks, it’s always an interesting and useful read. Be sure to follow Rebecca on Twitter because she is great at microblogging too.
- SE hipster is another great blog, but there you will always find some unique hints and tricks for your android app.
- Grafix Artist is an awesome source for UI and UX materials. A lot of examples and uncommon ways to solve problems, plus a bunch of great pictures.
- Android Authority is similar to the previously mentioned 9to5mac but contains fresh news and reviews about Android. It can be sorted by country, or you can catch up on what’s popular globally.
General Mobile Development Resources
Here are a few resources for mobile development which could help with testing various mobile applications via trends and news from non-development teams like project managers and public relations managers.
- Mobile Dev Memo offers marketing and analytics articles which can help you to grow as a project manager. This Appodeal article on "How to deal with COPPA", for example, introduces you to the legal side of apps, how it can change the whole idea of the product, and what you should avoid when working with CLID (Calling Line Identification) data. Also, you should check outthis article on the most popular brands among different user bases. It will probably help you to build your device farm more carefully.
- Savvy apps is another brilliant resource with a bunch of inspiring articles for everyday mobile app development. One of my personal favourite examples is the article about REST API. It's an easy and fast introduction to the whole theme without any code (almost). Also, a great read is this article about iOS 11 features and why you should implement them inside your app. The latter article is good for generating discussion about new iOS features with your team.
Mobile Events List
Another way to access beneficial information is to go to mobile events. Here is a list of mobile events which can be interesting to attend and useful for a mobile-centric tester.
- Apple’s semi-annual conference. Every autumn Apple shows the world new iPhones and every spring new iOS, macOS and other software stuff. The iPhone presentation is more about hardware and it’s new features. The autumn software presentation (e.g. WWDC) is all about new tools, which include such things as new functions inside Apple's software products, new methods inside testing and developers tools, and new operating systems for Mac and iPhone.
- Google I/O is an annual conference about software and hardware that Google hosts for their developers and customers. At this conference, you can find information about new testing tools, operating systems, and design patterns.
- ADDC annual conference. It is random when they happen, but they are always interesting. Here you can find inspiring speeches from developers and designers. You can also find new approaches in development and creative tools for UI and UX.
- Developer Week is another independent conference with a bunch of themes focused on developers and trends inside the IT sphere.
Keep In The Loop
As you can see, mobile app testing is a vast and interesting area, but you should be careful, active and highly motivated to stay effective. I hope this article will help you acquire new knowledge to set you on your way, that it will be handy to refer to for the foreseeable future and will bring something new with every read. But, due to the fast-paced nature of the mobile testing space, there are new things to know almost on a weekly basis and lots more pro tips to discover. I’d like to hear your thoughts on testing mobile and of any tips or tools you can recommend over at The Club.
Max is a certified psychologist who has found himself in testing. In the last three years, he’s managed to be a manual tester, write code for a couple of small web projects, grow into test automation and, finally, lead the testing department at mobile development studio Trinity Digital.