XCode shouldn't tell you that your tests have run when actually otest crashed

Originator:sam
Number:rdar://13063570 Date Originated:22-Jan-2013 08:56 PM
Status:Open Resolved:
Product:Xcode Product Version:Xcode Version 4.6 (4H112f)
Classification:UI/Usability Reproducible:Always
 
Summary:

For full details, see this blog post:
http://www.bornsleepy.com/bornsleepy/mysterious-case-unit-tests-ran-speed-light

Essentially, when a unit test bundle fails to load because of a missing shared library, otest crashes and displays some spurious advice about garbage collection.

XCode then compounds this problem by presenting the user with a UI which suggests that all tests have run successfully.

Steps to Reproduce:

Make a unit test that relies on a custom shared library which isn't present when the test runs.

Expected Results:

Xcode should report that your tests failed to run. If possible it should indicate that the unit test bundle failed to load because of a missing library. In an ideal world it should tell you which library, and which library was trying to load it (ie it may be a dependency of a dependency).

Actual Results:

Xcode reports that otest crashed, but only in the console. In the Log Navigator Xcode shows a successful (green) banner, even though no tests actually ran.

Regression:

Notes:

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!