Update
The information below is mostly redundant. After filing a bug report with Apple, their engineers determined that the Xcode-detected set of macro actions (find a text field, double click, enter text) weren’t working because the double click action wasn’t editing the text field. It is possible to use UIAutomation Tests, you just have to carefully review the UI actions and determine that they have the effect expected, particularly after letting Xcode record UI macros.
Original Post
Unfortunately my work to organise UIAutomation tests has hit the stumbling block that the UI Automation runner doesn’t use the main thread for main-thread-only APIs.
In Xcode 9 and High Sierra, the authors of that post I just linked found that it was possible to turn off the main thread checker in the Test configuration of the build scheme and get working tests anyway. Unfortunately that doesn’t work for me in Xcode 10 and Mojave: the main thread checker isn’t killing the app: the TSM subsystem is just refusing to do its thing. So my tests can’t do straightforward things like write text into a text field. Unfortunately this is a “it’s not me, it’s you” moment, and I don’t think I can carry on using Xcode’s UI tests for my goals.
However, I still want to be able to write “end-to-end” level tests to drive my development. I have (at least) three ways to proceed:
- I could find a third party library and discover whether it has the main thread problem. Calabash doesn’t support Mac apps, and the other examples I can find (Cucumberish and TABTestKit) both rely on UI Automation so presumably don’t address the main thread problem.
- I could write the tests in AppleScript. That would be a good way to build up the AppleScript UI for the app, but it doesn’t represent an end-to-end test of the GUI.
- I could write the tests using
NSApplication.sendEvent(_ event:)to simulate clicks, scrolls and text entry, and use the unit test runner to host them.That could work, but I have my doubts (I would guess that the runner is synchronous and stalls the main thread).
I discovered that it is possible to write the test “at the UI level” but in the unit runner, using a combination of key events and AppKit API like sendAction( to:). The trade-offs of this approach:
- it takes longer, as the abstractions needed to easily find and use AppKit controls don’t (currently) exist
- it doesn’t use the Accessibility interface so isn’t an accessibility audit at the same time as a correctness test
- you don’t hit the same problems as with the UI Automation runner
- it’s much faster
This may be the best approach for now, though I’d welcome other views.
