I am sure everybody has heard about Cucumber ( https://github.com/aslakhellesoy/cucumber) – a tool for Behaviour Driver Development where you describe software behavior in natural language that your customer can understand. Through step definitions these behavior descriptions are executed as automated tests. Cucumber serves as documentation, automated tests and development aid.
My friend and colleague Christian Hedin gave me tips on iCuke. Cucumber has been widely used for testing web applications, but now it’s also possible to test iOS (iPhone and iPad) apps with help of the iCuke library (https://github.com/unboxed/icuke). iCuke uses AppleScript to drive XCode in order to launch your application into the iOS Simulator. A preloaded library is used to inject a small HTTP server into your application. The HTTP server allows you to see an XML representation of the iOS device screen and to emulate input, such as taps, swipes and pinch gestures.
You can read more about it here:
http://www.unboxedconsulting.com/blog/cucumber-iphone-icuke
http://pragprog.com/magazines/2010-07/bdd-on-iphone-icuke
Really excited about Cucumber on iPhone we decided to give it a try. After installing iCuke and doing some test runs it was clear that some challenges had to be overcome to make this a truly useful tool for Behaviour Driven Development and automated testing of iOS apps.
Challenge 1: Screen returns the previous screen's xml
In our test, we wanted to tap a button, come to another screen and expected to see a text. When the test runs the iPhone is driven to the correct screen, e can see the expected text but the test fails anyway. After little debugging we realized that the method screen.xml returns old xml directly after changing the screen.
Solution:
We needed to refresh screen before checking if the expected text is on the new screen. iCuke has a method to refresh screen but it is private and we could not use it. So we just added a new method to the existing ICukeWorld class.
class ICukeWorld
def refresh_screen
refresh
screen
end
end
Calling this method before checking for the presence of the text solved this problem.
Later on, I forked iCuke and added this and some other methods.
git://github.com/DavorC/icuke.git
Challenge 2: Timing
Everybody knows that using sleep and delays in code is not so flexible.
sleep 3 # wait 3 seconds
We want to check something on the screen and give it 3 seconds to finish its loading.
Is it enough? Maybe, maybe not. Screen content loading could take 0.1 second or 5 seconds or... - You know what I mean.
In the first case loading is finished quickly and we unnecessarily spend 3 seconds for doing nothing . If we have a lot of delays in our code then our tests would waste a lot of precious time.
In the second case the delay is not long enough and the test fails.
Solution:
We need to write some help functions to wait for different items which are expected: some text, a button, downloading spinner etc.
Example with wait for text:
def wait_for_text(text, timeout = @@timeout)
puts "#{method_name}(#{text}, #{timeout})" if @@debug
refresh_screen
start_time = Time.now
until(screen.exists?(text)) do
if Time.now - start_time > timeout
flunk("#{method_name}: Timed out after #{timeout} seconds")
end
sleep 0.1
refresh_screen
end
end
As you can see the method is waiting for the text to appear and does checking every 0.1 second. As soon as the text is found the test continues. If, after given timeout, the text is still not found, the test fails.
I prefer to use unit test assertions in my tests (flunk is an assert which always fails). To use assertions with cucumber you need to add assertions to the Cucumber World:
require 'test/unit/assertions'
World(Test::Unit::Assertions)
Challenge 3: Different tappable object on screen can have the same text label.
Identifying objects on the screen only by text is not enough. By default, the first tappable object is tapped. What if we want to tap the second one?
Solution:
I created a set of help functions for:
returning all objects that satisfy some criteria
returning a specific object
waiting for a specific object
checking if a specific object exists
Objects are described in xml by: type, label, traits and index.
Here is an example of getting an array of all elements satisfying given attribute values.
def get_all_elements_by_type_label_and_traits(type, label, traits)
puts "#{method_name}(#{type}, #{label}, #{traits})" if @@debug
refresh_screen
doc = REXML::Document.new(screen.xml.to_s)
elements = REXML::XPath.match(doc, "//#{type}[@label=#{label.inspect}][@traits=#{traits.inspect}]")
elements
end
Observe how it is easy to parse xml using ruby's REXML library.
Of course I could write more generic methods and decrease number of code lines - something like:
get_all_elements(type, options = {})
but I like readability so I wrote a set of help functions with more specific naming:
get_element_by_type(type, index = 0)
get_element_by_type_and_label(type, label, index = 0)
get_element_by_type_and_traits(type, traits, index = 0)
get_element_by_type_label_and_traits(type, label, traits, index = 0)
get_all_elements_by_type(type)
get_all_elements_by_type_and_label(type, label)
get_all_elements_by_type_and_traits(type, traits)
get_all_elements_by_type_label_and_traits(type, label, traits)
get_all_static_texts()
get_all_labels_by_type(type)
get_all_labels_by_traits(traits)
element_by_type_exists?(type, index = 0)
element_by_type_and_label_exists?(type, label, index = 0)
element_by_type_and_traits_exists?(type, traits, index = 0)
element_by_type_label_and_traits_exists?(type, label, traits, index = 0)
text_exists?(text)
wait_for_element_by_type(type, index = 0, timeout = @@timeout)
wait_for_element_by_type_and_label(type, label, index = 0, timeout = @@timeout)
wait_for_element_by_type_and_traits(type, traits, index = 0, timeout = @@timeout)
wait_for_element_by_type_label_and_traits(type, label, traits, index = 0, timeout = @@timeout)
wait_for_text(text, timeout = @@timeout)
get_center_of_the_element(element)
tap_coordinates(x, y)
double_tap_coordinates(x, y)
tap_element(element)
double_tap_element(element)
tap_text(text)
double_tap_text(text)
In order to use these functions you need to use iCuke from:
git://github.com/DavorC/icuke.git
Don't forget to use –recursive flag when you clone it:
git clone --recursive git://github.com/DavorC/icuke.git
After building and installing the iCuke gem you need to
require 'icuke/cucumber_ext'
instead of:
require 'icuke/cucumber'
Example of usage
In your feature-file:
Background:
Given "myApp.xcodeproj" is loaded in the simulator
Scenario Outline: User try to login with different invalid credentials with valid signs
When I am in "Account" section
And I paste in username "<user>"
And I paste in password "<pass>"
And I tap Login button
Then I will see alert dialog
Examples:
| user | pass |
| test@test.com | test |
| 123456 | qwertyui |
| ..@..com | . . . €<> |
step definitions:
When /I am in "(.*)" section/ do |section|
wait_for_text(section)
tap(section)
wait_for_element_by_type_and_label("UINavigationItemView", section)
end
When /I paste in username "(.*)"/ do |user|
label = "E-mail"
refresh_screen
assert(get_all_static_texts().include?(label), "No text field with label #{label} was found")
write_to_mac_clipboard(user)
paste_clipboard_to_text_field("UITextFieldLabel", label)
end
When "I tap Login button" do
wait_for_text("Login")
tap("Login")
wait_for_download_indicator_finish
end
Then "I will see alert dialog" do
wait_for_element_by_type("UIAlertView")
end
Testing both iPhone and iPad
If
you’re testing a universal app that runs on both iPhone and iPad I
recommend writing different scenarios for the platforms. iPad in
landscape mode is most likely to reuse most of code you have written for
iPhone. Place them in different feature files and tag them with e.g.
@iphone respective @ipad tags.
In your env.rb file:
$PLATFORM = "iphone"
Before('@ipad') do
$PLATFORM = "ipad"
end
Before('@iphone') do
$PLATFORM = "iphone"
end
Use in your feature files:
Given I have started application
Implementation:
Given "I have started the application" do
# ... some code
Given "\"myApp\" from \"myApp.xcodeproj\" is loaded in the #{$PLATFORM} simulator"
# ... more code
end
If you wants to run iPad simulator in landscape mode:
def switch_ipad_to_landscape
if(get_ipad_orientation == PORTRAIT)
rotate_simulator_left
end
end
where get_ipad_orientation is some application specific method to decide if the simulator is in portrait or landscape mode.
Running iCuke tests on Hudson server
It’s really nice to be able to run your test suite at given intervals, or when you commit to the source code repository. To run your iCuke tests on Hudson (which is a popular continuous integration build server) you must start your iPhone simulator from a terminal window. This is easiest to do by launching an AppleScript from Hudson.
Below are two scripts, one AppleScript and one shell script, that I used to run my iCuke tests from Hudson.
- run_cuke.scpt
-- run_cuke.scpt
tell application "Finder"
set my_folder_path to container of (path to me) as text
set posixPath to POSIX path of file my_folder_path
set scriptPath to posixPath & "cuke.sh"
end tell
tell application "Terminal"
activate
do script scriptPath
end tell
delay 1200 -- it should be enough to finish all tests
set logscript to "grep -c FAILED " & posixPath & "cuke.log" & " | cat"
set cuke_failed to do shell script logscript
if cuke_failed > 0 then
error "Cucumber tests failed"
end if
try -- do not leave terminals after test run
do shell script "killall 'Terminal'"
end try
- cuke.sh
#!/bin/bash
scriptpath=$(cd ${0%/*} && echo $PWD/${0##*/})
SCRIPTFOLDER=`dirname "$scriptpath"`
echo Cucumber script is run in: $SCRIPTFOLDER
cd $SCRIPTFOLDER
cucumber $SCRIPTFOLDER/features --format=html --out $SCRIPTFOLDER/cuke_results.html > $SCRIPTFOLDER/cuke.log
Add these two scripts to your project.
Then, add this Cucumber hook to your env.rb file:
After do |s|
if(s.failed?)
puts "Scenario FAILED: <#{s.name}>"
puts "More info about failure: SCREEN XML"
puts screen.xml
else
puts "Scenario PASS: <#{s.name}>"
end
end
On Hudson, create a new job and copy the configuration from your project's existing job.
Configure job:
add to description:
<a href="/hudson/job/Helios_Cuke/ws/cuke_results.html">Cucumber Results</a>
<b> | </b>
<a href="/hudson/job/Helios_Cuke/ws/cuke.log">Debug.log</a>
add build step (execute shell):
osascript ${WORKSPACE}/run_cuke.scpt
Of course you can change all scripts according your needs.
At
the end you should have your iCuke tests running on Hudson and you will
get both a nice test report in HTML and the debug output as a plain
text.
There
is a lot of potential for doing automated feature tests for iOS using
Cucumber and with iCuke and the additions above you’ll hopefully be well
on your way for doing BDD in your next iOS project!
转自:http://blog.jayway.com/2011/02/11/cucumber-tests-on-iphoneipad/