Xcode UI Testing: XCUI tests need to be able to access the application process

Originator:designatednerd
Number:rdar://23116258 Date Originated:14-Oct-2015 05:01 PM
Status:Duplicate Resolved:Duplicate
Product:Developer Tools Product Version:Xcode 7.0.1 (7A1001)
Classification:UI/Usability Reproducible:Always
 
Summary:
XCUI tests are currently run in a completely separate process than the app under test, which makes a lot of sense in theory: You want to test the UI, so you only give the user access to the application proxy. 

In practice, this is a bit of a disaster, because you can’t mock anything out without having to set up a ton of stuff on the app side. I can’t tell my UI tests “Test what happens when the user tries to login with network connectivity” and “Test what happens when the user tries to login without network connectivity” because those things are being determined by the app itself, which I have no access to. 

Steps to Reproduce:
1. Write some UI tests. 
2. Try to @testable import your app bundle, add a few things from your app bundle like string convenience methods
3. Get error about unsafe mutable accessors, scratch head
4. Google it, discover that because test is run in a separate process testable importing doesn’t work for UI tests. 
5. Realize that this means you can’t set up any kind of helpful state like “Use a mock network connection” or “Let’s pretend the user tries to do x without a working internet connection”

Expected Results:
Be able to access the application process to update these types of objects within the main application at runtime.

Actual Results:
Any state must be set up using environment arguments in a build scheme, and coded into the application rather than set up in the test target. 

Regression:
UI testing was just introduced, but it’s a “regression” from the standard XCTest suite which allows access to these things. 

Notes:
This is a major dealbreaker for me in actually using XCUI testing because if I can’t mock state, then I’m testing not only UI but the exact state of the app at any time. Essentially, I don’t want my UI tests to fail because my device cabinet is in a part of the office with iffy wifi.

Comments

That was fast: "Engineering has determined that your bug report is a duplicate of another issue and will be closed."

By designatednerd at Oct. 15, 2015, 11:46 p.m. (reply...)

Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!