Wether you like to eat zucchini or not, you have to take a look at this tasty testing framework for iOS. Over time I’ve looked at a lot a testing frameworks for iOS. Now I came across this new framework for interface testing.
It’s not some completely new fancy way to test, you still have to write tests and run them. But it splits the tests in two very useful parts. In a part where you write what you want to test in a more or less natural language. And in another part where you define screens and what features every screen has. So these screens are reusable for every test.
During the test you can take a screenshots. At the end of the test these taken screen shots are presented in an HTML report. There you can see the diff between the captured image and a reference image you provided previously. You can even define some masks which part of the screenshot should not be compared. E.g. the menu bar. So a change in the time does not break your test.
While I was setting zucchini up to run with my build server I stumbled up on some problems. First of all there was an issue with Instruments being run from within Java or Jenkins. It just hang forever and endlessly leaked memory. I didn’t figure out what the problem exactly was. I ended up with running the rake file from Jenkins over SSH. So the execution of Instruments is decoupled from the Jenkins thread.
Another problem is that the path to the binary that should be deployed and tested is not configurable on runtime. It’s configured in a configuration file but can’t be overridden when running zucchini. So before starting the test you have to copy the binary to the configured location. Maybe that feature will be integrated in a future release. The same problem exists with the generated report. The report is just opened in Safari. It would be nice if an output path for the report could be configured. So an archive of all reports for every run could be made.
Overall it looks like a very promising testing framework and maybe one ore another feature will be integrated in future releases.