Problem:
Are there any situations in which MobileApp testing on actual devices can be disregarded?
Answer:
Testing is carried out to uncover errors such that a quality product is delivered to the consumer. Ideally testing can never be ignored or skipped. A mobileApp generally is tested on mobile devices to make sure that it does not result malfunctioning once it is released to devices. Other than that, it is tested on a variety of devices to make sure that it runs smoothly on all the devices capable of running that mobileApp.
However, if a mobileApp is designed to execute completely on a mobile device, its testing on an actual device can be skipped. Rather, it can be tested on emulators on computer systems. Such a mobileApp can also be tested using the traditional approaches of software testing. Skipping the testing on mobile devices does not mean skipping the testing all together. It only means that rather than testing on actual devices, the mobileApp can be alternatively tested. Thus a mobileApp can be tested on emulators if it is meant to operate fully on a mobile device.
Problem:
In your own words, discuss the objectives for testing a MobileApp.
Answer:
Like WebApp, the MobileApp has several dimensions for quality. The main goal of testing MobileApp like testing any other application is to uncover issues and find errors that would lead to one or many quality features. The different objectives for testing a MobileApp are listed below:
• Unit testing begins for testing the content of MobileApp followed by testing its functionality and navigation.
• After this validating the individual units, the integrated MobileApp testing should commence.
• Generally the MobileApp is tested from a user’s perspective. Test cases are derived to uncover inefficiencies that my affect a user’s experience.
• The testing of content helps to uncover errors in presentation of the content on a MobileApp. This makes sure that the end user does not face any content related performance issues that may arise for constraints on a mobile device.
• There may be different mechanism in interaction especially if the mobile device is not taken into consideration for a MobileApp. Thus it is also tested to uncover errors on interaction related issues it may face while the user is using it.
• Compatibility is a major issue as the MobileApp may be designed to work on all platforms. It should be tested in every aspect on different platforms so to errors related to each platform can be uncovered.
• MobileApp may use some user specific data which must be kept secure at all costs. Security testing carries out the work of uncovering security related errors.
• Server responsible of responding to a MobileApp is performance tested to find errors that may arise due to limited server resources.
• In all, MobileApp is also tested to find performance related issues such as the power consumption, the computation speed, limitations of memory, failure recovery, issues in connectivity.
These are the major important objectives for testing a MobileApp.
Problem:
Compatibility is an important quality dimension. What must be tested to ensure that compatibility exists for a MobileApp?
Answer:
Compatibility is an important quality dimension as a MobileApp may be designed to work on different platforms and devices of different configuration. It should be tested in every aspect on different platforms so that errors related to each platform can be uncovered. Following tests must be performed to ensure compatibility exists for a particular mobileApp:
• The mobileApp should be functional on different operating system environments.
• It should display quality content and navigational features to the user.
• All features should be same and fully functional across all platforms.
• The response time of a mobileApp should be approximately similar.
• Devices with different hardware configuration should also be tested thoroughly to check the support of mobileApp.
These are few tests that can help get an insight of the compatibility testing of a mobileApp.
Problem:
Locate a free MobileApp testing tool. Critique the performance of the tool relative to the MobileApp with which you are familiar.
Answer: Problem:
What elements of the MobileApp can be “unit tested”? What types of tests must be conducted only after the MobileApp elements are integrated?
Answer:
The main goal of testing MobileApp like testing any other application is to uncover issues and find errors that would lead to one or many quality features. Unit testing begins for testing the content of MobileApp followed by testing its functionality and navigation. The elements of a mobileApp which can be unit tested are given below:
• Content: The testing of content helps to uncover errors in presentation of the content on a MobileApp. This makes sure that the end user does not face any content related performance issues that may arise for constraints on a mobile device.
• Functionality: The functionality tests help in uncovering errors in the basic operations of the mobileApp.
• Navigation: Generally the MobileApp is tested from a user’s perspective. Test cases are derived to uncover inefficiencies that my affect a user’s experience. Switching from one page to another, scrolling down and up etc is all tested.
After validating the individual units, the integrated MobileApp testing should commence. The different tests that must be conducted once elements of mobileApp are integrated are given below:
• Configuration testing: Compatibility is a major issue as the MobileApp may be designed to work on all platforms. It should be tested in every aspect on different platforms so to errors related to each platform can be uncovered.
• Security testing: MobileApp may use some user specific data which must be kept secure at all costs. Security testing carries out the work of uncovering security related errors.
• Performance testing: MobileApp is also tested to find performance related issues such as the power consumption, the computation speed, limitations of memory, failure recovery, issues in connectivity.
As a part of integration testing the mobileApp is tested as a whole using these testing strategies.
Problem:
Is it always necessary to develop a formal written-test plan? Explain.
Answer:
A written test plan is always a guide to the process of uncovering errors in different aspects of software testing. It helps in keeping a record of all the tests that need to be run on system. Since it is developed as the project progresses, it is easier to record many possible loop holes that need to be tested. A written test plan documents ideas at high level design regarding testing. The test plan can project major tests needed for the software beginning from the requirements phase of the system itself.
• A formal written test plan outlines a process for unit testing and integration testing.
• It also define the required test points in the testing phase.
• Once testing is completed, it can be regarded as an important resource to refer to the testing that was carried out for a system.
• It also serves as a useful resource for regression testing.
Thus it is important to develop a written test plan.
Problem:
Is it fair to say that the overall MobileApp-testing strategy begins with user-visible elements and moves toward technology elements? Are there exceptions to this strategy?
Answer:
Since mobileApps are developed for end users who are the key players in determining whether a mobileApp is really useful or not it is very important to design user-friendly good performance mobileApps. When testing mobileApps, it usually begins with user-visible elements such as content, interface and, navigation. This is primarily done to understand whether the mobileApp developed will be able to deliver expected content and interfaces to the user or not. Once the user-visible components are tested and corrected for errors, testing can be extended to technology and infrastructure. This is where the actual functioning and performance of the mobileApp are tested. Thus it is pretty fair to say that testing begins user-visible elements and later proceeds to the technological aspects.
It all basically comes down to the development phases when we think of exceptions to this strategy. For instance, if a prototype has been generated for a mobileApp showcasing its functionality and verifying other performance matrices, then certainly the testing is conducted for technology and not user-visible aspects. Therefore, there can be exceptions to testing mobileApps from traditional approaches.
Problem:
Is certification testing really testing in a conventional sense? Explain.
Answer:
Certification testing checks whether a MobileApp meets all the requirements and standards that are established by the app stores which ultimately distribute the MobileApp to users. App stores for different platforms have different set of standards and protocols which each MobileApp must fulfil. Few of these standards help in testing as well but certification testing cannot be considered as testing in a conventional sense.
Certification testing is just a benchmark for a MobileApp to be available in the app stores. It does not make a MobileApp free of errors. Errors may persist in a MobileApp if proper testing has not been carried out. The MobileApp may face serious problems even if certification testing has been performed but not the real conventional testing making it improper to use for users.
Thus, certification testing should only be regarded as a final step of testing after all other testing has been carried out.
Problem:
Describe the steps associated with user experience testing for a MobileApp.
Answer:
User experience of a mobileApp is very crucial that it why it is often advisable to allow users in the testing cycle as soon as possible. Generally the MobileApp is tested from a user’s perspective. Test cases are derived to uncover inefficiencies that may affect a user’s experience. The testing of content helps to uncover errors in presentation of the content on a MobileApp. This makes sure that the end user does not face any content related performance issues that may arise for constraints on a mobile device.
Most users expect that the mobileApp they are considering is context aware and also capable of delivering personalised experiences depending on the device’s physical location and network features. Different steps that are associated with testing of a mobileApp with respect to user experience are given below:
• Testing whether the mobileApp can be used in different countries and different languages.
• Testing the mobileApp in unpredictable and adverse environments.
• Testing the mobileApp using specific hardware.
• Testing the mobileApp using outdated or less-used browsers.
• Testing the mobileApp using unresponsive plug-ins.
• Testing the mobileApp using different set of devices.
• Testing the mobileApp using imperfect connectivity.
These are few testing steps that can be included while performing user experience testing.
Problem:
Assume that you are developing a MobileApp to access an online pharmacy that caters to senior citizens. The pharmacy provides typical functions, but also maintains a database for each customer so that it can provide drug information and warn of potential drug interactions. Discuss any special usability tests for this MobileApp.
Answer:
For a MobileApp that maintains a database for each user to provide them information about drugs and possible drug interactions, several usability tests can be thought of. For instance, the very first and basic usability test is whether it is able to produce the entire list of drugs a consumer has ordered. This is a basic but a very important test as this list contains the names of drugs and their purchase date which can be used to find what illness has a consumer suffered over a period of time.
Another test can help in finding whether there can be a single drug rather than two separate drugs. For example there are iron tablets and there are folic acid tablets. If a consumer orders these two, the MobileApp can suggest one single tablet for iron and folic acid.
For a general purpose, the MobileApp can display which drug has been mostly used by its consumers. Identifying the drug can help in identifying the disease associated with it. As a complimentary feature the MobileApp can suggest some changes in lifestyle to avoid that disease to all its consumers.
Problem:
Assume that you have implemented a Web service that provides a drug interaction– checking function for YourCornerPharmacy.com (see Problem 26.10). Discuss the types of component-level tests that would have to be conducted on the mobile device to ensure that the MobileApp accesses this function properly.
Answer:
For a MobileApp that interacts with a web service to provide information to its users several component-level tests should be conducted to ensure that it functions properly. Few such tests are given below:
• Test how the MobileApp will interact with the web service. This is usually done using authentication at the server-side.
• Test the response time of the web service to provide drug interaction details back to the MobileApp and check whether it is in acceptable limits.
• Test what happens if the network connectivity is lost while the MobileApp tries to access the web service to cater its customers.
• Check whether correct information has been sent by the web service. Also check whether only needed information has been received by the MobileApp.
• Check the content, context and interface appearance of the data received by the MobileApp. Check whether it is neatly displayed.
These component-level tests help to find whether the web service is functioning properly and whether the MobileApp is efficiently able to provide the data it has received.
Problem:
How can a MobileApp’s ability to take context into account be tested?
Answer:
The testing of content helps to uncover errors in presentation of the content on a MobileApp. This makes sure that the end user does not face any content related performance issues that may arise for constraints on a mobile device.
There can be several configuration variants such as operating systems, devices and, mobile networks. A MobileApp may encounter different combinations of these configuration variants. In such a scenario, testing should be thoroughly carried out to make sure that the MobileApp is completely functional and also context aware.
Thus automation testing can be used in this scenario to check the efficiency of the MobileApp. Although not all parts of MobileApp can be automated to test, automation testing is a very important tool to consider a MobileApp’s capability to be context aware.
Problem:
Is it possible to test every configuration that a MobileApp is likely to encounter in the production environment? If it is not, how do you select a meaningful set of configuration tests?
Answer:
Testing strategies have evolved over time. Most testing is carried out in the production environment. Although the production environment attempts to uncover all errors in software, they may lack few holes in the system. Compatibility is a major issue as the MobileApp may be designed to work on all platforms. It should be tested in every aspect on different platforms so to errors related to each platform can be uncovered. Configurations of the device running the MobileApp may also differ from user to user and country to country. In such a scenario it becomes impossible for the production team to test each and every possible configuration that a MobileApp might encounter.
Thus to be able to successfully cover a meaningful set of configuration tests third party vendors can be outsourced who can perform these configuration tests. These tests can then be conducted by the outsourced team to uncover maximum possible errors.
Problem:
Describe a security test that might need to be conducted for the YourCornerPharmacy (Problem 26.10) MobileApp. Who should perform this test?
Answer:
For a MobileApp that maintains a database for each user to provide them information about drugs a security test that can be carried out is not to let an intruder access the information of user’s information stored in the database. The information can be sensitive or non-sensitive but no user would like to allow someone else access his personal or medical information. To disallow this intrusion, proper authentication should be used at the database server using strong credential requirements.
This test can be conducted by the development team itself but it is much better to hire a third party team to carry out security tests. The third party can use ethical hackers to check whether the security of this MobileApp is vulnerable.
Problem:
What is the difference between stress-testing a MobileApp and stress-testing a WebApp?
Answer:
Stress testing is uncovering errors in a program in extreme conditions. The approach is similar for stress-testing a MobileApp and a WebApp i.e. simply stressing the system to extreme levels and then checking its behaviour. Major differences between stress-testing a MobileApp and a WebApp are given below:
• Several MobileApps are run on the same device to stress-test a MobileApp, however running several WebApps does not stress-test a WebApp.
• The software of the system can be infected with viruses to stress-test a MobileApp whereas infecting the system does not become a part of stress-testing a WebApp.
• A MobileApp is stress-tested to check its device is capable of spreading spam. This does not apply to a WebApp.
Besides these a MobileApp and WebApp are stress-tested in many similar ways.
Solution: Chapter 26: TESTING MOBILE APPS
26.1 No. Mobile devices are very diverse in their display and input facilities. A MobileApp running on a PC emulator may or may run correctly on a particular device.
26.2 Testing is the process of exercising software with the intent of finding (and ultimately correcting) errors. The goal of MobileApp testing is to exercise each of the many dimensions of MobileApp quality with the intent of finding errors or uncovering issues that may lead to quality failures. It is particularly important to ensure that the MobileApp meets its non-functional requirements. This includes taking advantage of context to provide a good user experience.
26.3 Compatibility is tested by executing the MobileApp in a variety of different mobile device configurations and in a variety of real world networking environments.
26.4 Answers will vary,
26.5 Once each interface mechanism has been “unit” tested, the focus of interface testing changes to a consideration of interface semantics. Interface semantics testing “evaluates how well the design takes care of users, offers clear direction, delivers feedback, and maintains consistency of language and approach.” The elements of a MobileApp that can be unit tested are the components (e.g. scripts and web services) that deliver MobileApp functional units.
26.6 Testing shouldn’t wait until the project is finished. Start testing before you write one line of code. The intent is to uncover and correct errors before the MobileApp is made available to its end-users. A MobileApp test plan identifies (1) the task set to be applied as testing commences; (2) the work products to be produced as each testing task is executed, and (3) the manner in which the results of testing are evaluated, recorded, and reused when regression testing is conducted. Hence Formal written test plan is usually necessary for crucial applications.
26.7 The MobileApp testing strategy exercises each quality dimension by initially examining “units” of content, functionality, or navigation. Once individual units have been validated, the focus shifts to tests that exercise the MobileApp as a whole. To accomplish this, many tests are derived from users perspective and are driven by information contained in use-cases. An engineering test plan is developed and identifies testing steps, work products (e.g., test cases), and mechanisms for the evaluation of test results. The testing process includes testing in the wild.
26.8 Certification is often required if you wish to place you MobileApp in commercial app store. Some mobile platforms allow direct installation of a MobileApp either from web site of form your own PC.
26.9 MobileApp user experience testing include the following elements:
Functionality
Information architecture
Screen Design
User input mechanisms
Mobile context taken into account
Interface usability
Trustworthiness
Feedback
Help facilities
26.10 Answers will vary
26.11 Answers will vary
26.12 To check the MobileApp’s ability to take context into account, often involves testing by typical users, on ordinary mobile devices, and in geographically diverse locations.
26.13 It is not possible to test every possible MobielApp device configuration. It is important to a technique like a weighted device platform matrix to identify the most important device configurations to test for a particular set of user functions.
26.14 Answers will vary.
26.15 WebApps run on servers with fixed locations and established network. MobileApps need to be tested on ad hoc networks with a great variety of network and telephony type connections.