Under Xcode 5, there is no way to run logical tests without the simulator launching
| Originator: | markos | ||
| Number: | rdar://15859153 | Date Originated: | 20-Jan-2014 05:13 PM |
| Status: | Behaves correctly | Resolved: | |
| Product: | Developer Tools | Product Version: | 5.0.2 |
| Classification: | Other Bug | Reproducible: | Always |
Summary: Under Xcode 5 it should be possible to configure and run logic tests without the simulator launching as described in the (now retired) document "UnitTesting" under "Setting Up Logic Unit Tests" for Xcode 4. See https://developer.apple.com/legacy/library/documentation/DeveloperTools/Conceptual/UnitTesting/UnitTesting.pdf This is a pressing issue when it comes to automatically running logic tests as part of a build which is against the "red, refactor, green" cycle of test driven development and what the test navigator in Xcode 5 strives to achieve. Xcode 4 didn't have "launching the simulator" as a requirement when running logic tests and the now deprecated "Test After Build" elevated that promise. Steps to Reproduce: 1 Install Xcode 5.0.2 2 Create an "Empty Application" project 3 Select the "*Tests" target 3.1 Select "Build Settings" 3.2 Remove "Test Host" 3.3 Remove "Bundle loader" 4 Select the "*Tests" target 5 Select "Product", "Run" Alternatively using xcodebuild xcodebuild -project Bar.xcodeproj -scheme Bar -destination "platform=iOS Simulator,name=iPhone Retina (3.5-inch)" test Expected Results: Have the logic tests running without the simulator launching Actual Results: The simulator is launching as part of running logic tests Version: Xcode 5.0.2 Build version 5A3005 OS X 10.9.1 (13B42) Notes: "This is why Xcode launches the iOS Simulator even for logic tests — the tests need to actually run in an iOS environment to behave correctly." https://devforums.apple.com/message/867874#867874 "It sounds like you may want to test using the iOS Simulator, but not have the iOS Simulator app launch and show its GUI? If so, then that's not currently possible." https://devforums.apple.com/message/908014#908014 "@joar_at_work given your comment on the developer forums, is that a "known issue" or a deliberate change? Can still file a bug report if so." https://twitter.com/qnoid/status/412888557473107968 "How do I automatically perform unit tests on each “build and run” action in XCode 5?" http://stackoverflow.com/questions/20605509/how-do-i-automatically-perform-unit-tests-on-each-build-and-run-action-in-xcod Workaround: No known workarounds under Xcode 5 Configuration: Attachments: 'Bar.zip' was successfully uploaded.
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!
Apple Developer Relations 21-Jan-2014 04:02 PM
Engineering has determined that this issue behaves as intended based on the following information:
To execute iOS tests you need iOS. Logic tests are not supported for iOS devices at this time, so the only other option is iOS simulator. With Xcode 5 / iOS 7 and later, there's no way around launching the simulator app when using the simulator.
Here are some suggestions for other things you could ask for that may still address some of your concerns:
If you have questions regarding the resolution of this issue, please update your bug report with them.
We are now closing this bug report.
Please be sure to regularly check new Apple releases for any updates that might affect this issue.
Steps 4, 5 are wrongly specified. That should be replaced by: 4 Select "Product", "Test"
Which generates a transcript that includes: "Test Suite 'All tests' started at 2014-01-21 00:13:51 +0000"
Based on the now retired document, under "Listing 2-2 Transcript of an iOS Simulator-based application unit-test run", this log is related to "Setting Up Application Unit Tests".