Swiss Developer Immigrated to Germany

Tasty Zucchini

Wether you like to eat zucchini or not, you have to take a look at this tasty testing framework for iOS. Over time I’ve looked at a lot a testing frameworks for iOS. Now I came across this new framework for interface testing.

It’s not some completely new fancy way to test, you still have to write tests and run them. But it splits the tests in two very useful parts. In a part where you write what you want to test in a more or less natural language. And in another part where you define screens and what features every screen has. So these screens are reusable for every test.

Technically it’s based on the UIAutomation framework from apple. UIAutomation is integrated in Instruments and runs interface automations from JavaScript’s. I used to write many tests with it but was never really happy with it. There was always a lot a bloat code you had to write that made the tests break easily. The contact point between zucchini and UIAutomation is in these screens. They’re written in Coffee Script and map the features of a screen to UIAutomation. So the actual tests are decoupled from UIAutomation.

During the test you can take a screenshots. At the end of the test these taken screen shots are presented in an HTML report. There you can see the diff between the captured image and a reference image you provided previously. You can even define some masks which part of the screenshot should not be compared. E.g. the menu bar. So a change in the time does not break your test.

While I was setting zucchini up to run with my build server I stumbled up on some problems. First of all there was an issue with Instruments being run from within Java or Jenkins. It just hang forever and endlessly leaked memory. I didn’t figure out what the problem exactly was. I ended up with running the rake file from Jenkins over SSH. So the execution of Instruments is decoupled from the Jenkins thread.

Another problem is that the path to the binary that should be deployed and tested is not configurable on runtime. It’s configured in a configuration file but can’t be overridden when running zucchini. So before starting the test you have to copy the binary to the configured location. Maybe that feature will be integrated in a future release. The same problem exists with the generated report. The report is just opened in Safari. It would be nice if an output path for the report could be configured. So an archive of all reports for every run could be made.

Overall it looks like a very promising testing framework and maybe one ore another feature will be integrated in future releases.

Comments on: "Tasty Zucchini" (3)

  1. […] I’ve been playing around with the Zucchini Framework. I discovered a few pitfalls. Maybe someone else stumbles upon the same […]

  2. Tony Mann said:

    I just pushed a change to the Zucchini project that lets you specify the app bundle path via ZUCCHINI_APP. This will avoid the problem you mentioned with the config file.

  3. Tony Mann said:

    Another thing: I was able to get Instruments to launch via Jenkins by starting it /Library/LaunchAgents instead of LaunchDaemons. No ssh needed 🙂