Sqa Engineering Testing
Automating GUI Testing for Android Applications
Cuixiong Hu Iulian Neamtiu
Department of Computer Science and Engineering University of California, Riverside, CA, USA
{huc,neamtiu}@cs.ucr.edu
ABSTRACT
Users increasingly rely on mobile applications for computational needs. Google Android is a popular mobile platform, hence the reliability of Android applications is becoming increasingly important. Many Android correctness issues, however, fall outside the scope of traditional verification techniques, as they are due to the novelty of the platform and its GUI-oriented application construction paradigm. In this paper we present an approach for automating the testing process for Android applications, with a focus on GUI bugs. We first conduct a bug mining study to understand the nature and frequency of bugs affecting Android applications; our study finds that GUI bugs are quite numerous. Next, we present techniques for detecting GUI bugs by automatic generation of test cases, feeding the application random events, instrumenting the VM, producing log/trace files and analyzing them post-run. We show how these techniques helped to re-discover existing bugs and find new bugs, and how they could be used to prevent certain bug categories. We believe our study and techniques have the potential to help developers increase the quality of Android applications.
Categories and Subject Descriptors
D.2.4 [Software Engineering]: Software/Program Verification—Reliability; D.2.5 [Software Engineering]: Testing and Debugging—Testing tools,Tracing
General Terms
Reliability, Verification
Keywords
Test automation, Mobile applications, Google Android, GUI testing, Test case generation, Empirical bug studies
1. INTRODUCTION
Smartphones are becoming pervasive, with more than 195 million sold worldwide in the first three quarters of 2010
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
Cuixiong Hu Iulian Neamtiu
Department of Computer Science and Engineering University of California, Riverside, CA, USA
{huc,neamtiu}@cs.ucr.edu
ABSTRACT
Users increasingly rely on mobile applications for computational needs. Google Android is a popular mobile platform, hence the reliability of Android applications is becoming increasingly important. Many Android correctness issues, however, fall outside the scope of traditional verification techniques, as they are due to the novelty of the platform and its GUI-oriented application construction paradigm. In this paper we present an approach for automating the testing process for Android applications, with a focus on GUI bugs. We first conduct a bug mining study to understand the nature and frequency of bugs affecting Android applications; our study finds that GUI bugs are quite numerous. Next, we present techniques for detecting GUI bugs by automatic generation of test cases, feeding the application random events, instrumenting the VM, producing log/trace files and analyzing them post-run. We show how these techniques helped to re-discover existing bugs and find new bugs, and how they could be used to prevent certain bug categories. We believe our study and techniques have the potential to help developers increase the quality of Android applications.
Categories and Subject Descriptors
D.2.4 [Software Engineering]: Software/Program Verification—Reliability; D.2.5 [Software Engineering]: Testing and Debugging—Testing tools,Tracing
General Terms
Reliability, Verification
Keywords
Test automation, Mobile applications, Google Android, GUI testing, Test case generation, Empirical bug studies
1. INTRODUCTION
Smartphones are becoming pervasive, with more than 195 million sold worldwide in the first three quarters of 2010
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that
References: [1] andoku. http://code.google.com/p/andoku/. [2] android-cmis-browser. http://code.google.com/p/android-cmis-browser/. [3] android-delicious-bookmarks. http://code.google. com/p/android-delicious-bookmarks/. [4] connectbot. http://code.google.com/p/connectbot/. [5] dealdroid. http://code.google.com/p/dealdroid/. [6] guessthenumber. http://code.google.com/p/guessthenumber/. [7] monolithandroid. http://code.google.com/p/monolithandroid/. [8] opensudoku-android. http://code.google.com/p/opensudoku-android/. [9] rokon. http://code.google.com/p/rokon/. [10] skylight1. http://code.google.com/p/skylight1/. [11] T. J. Watson Libraries for Analysis (WALA). http:// wala.sourceforge.net/wiki/index.php/Main_Page. [12] Android activity lifecycle, May 2010. http://developer.android.com/reference/ android/app/Activity.html#ActivityLifecycle. [13] Google code, April 2010. http://code.google.com/. [14] Guitar – a gui testing framework, August 2010. http://guitar.sourceforge.net/index.shtml. [15] JUnit, May 2010. http://www.junit.org/. [16] Monkey UI/Application Exerciser, May 2010. http://developer.android.com/guide/developing/ tools/monkey.html. [17] Androlib. Number of New Applications in Android Market by month, September 2010. http://www.androlib.com/appstats.aspx. [18] A. Chaudhuri. Language-based security on android. In PLAS ’09, pages 1–7. [19] W. Enck, P. Gilbert, B. gon Chun, L. P. Cox, J. Jung, P. McDaniel, and A. N. Sheth. Taintdroid: An information-flow tracking system for realtime privacy monitoring on smartphones. 2010. [20] W. Enck, M. Ongtang, and P. McDaniel. Understanding Android security. IEEE Security and Privacy, 7(1):50 –57, Jan.-Feb. 2009. [21] D. Engler and K. Ashcraft. Racerx: effective, static detection of race conditions and deadlocks. SOSP, pages 237–252, 2003. [22] Gartner Corporation. Gartner Says Worldwide Mobile Device Sales Grew 13.8 Percent in Second Quarter of 2010, But Competition Drove Prices Down, August 2010. http://www.gartner.com/it/page.jsp?id=1421013. [23] Gartner Corporation. Gartner Says Worldwide Mobile Phone Sales Grew 17 Per Cent in First Quarter 2010, May 2010. http://www.gartner.com/it/page.jsp?id=1372013. [24] Gartner Corporation. Gartner Says Worldwide Mobile Phone Sales Grew 35 Percent in Third Quarter 2010; Smartphone Sales Increased 96 Percent, November 2010. http://www.gartner.com/it/page.jsp?id=1466313. [25] D. R. Hackner and A. M. Memon. Test case generator for guitar. In ICSE Companion ’08, pages 959–960. [26] A. Kervinen, M. Maunumaa, T. P¨¨kk¨nen, and aa o M. Katara. Model-based testing through a gui. In Formal Approaches to Software Testing, volume 3997, pages 16–31. 2006. [27] A. Kumar Maji, K. Hao, S. Sultana, and S. Bagchi. Characterizing failures in mobile oses: A case study with android and symbian. In Software Reliability Engineering (ISSRE), 2010 IEEE 21st International Symposium on, pages 249–258. [28] M. Ongtang, S. Mclaughlin, W. Enck, and P. Mcdaniel. Semantically rich application-centric security in android. In ACSAC’09: Annual Computer Security Applications Conference, 2009. [29] M. Swift. Android operating system is expected to surge past rivals. Los Angeles Times, Sep 11 2010. [30] X. Yuan and A. M. Memon. Generating event sequence-based test cases using gui runtime state feedback. IEEE Trans. on Software Engineering, 36:81–95, 2010.