Open Radar

 
Peter Steinberger19-Dec-2016 05:59 PM

Summary:
We're testing a scheme with multiple test targets. Some test targets run, until one encounteres the following error:
2016-12-19 11:15:17.060 xcodebuild[4519:1006396] Error Domain=FBSOpenApplicationServiceErrorDomain Code=1 "The request to open "com.pspdfkit.coretests.ios" failed." UserInfo={NSUnderlyingError=0x7f8204ca9a00 {Error Domain=FBSOpenApplicationErrorDomain Code=6 "Application "com.pspdfkit.coretests.ios" is installing or uninstalling, and cannot be launched." UserInfo={BSErrorCodeDescription=Busy, NSLocalizedFailureReason=Application "com.pspdfkit.coretests.ios" is installing or uninstalling, and cannot be launched.}}, NSLocalizedDescription=The request to open "com.pspdfkit.coretests.ios" failed., NSLocalizedFailureReason=The request was denied by service delegate (SBMainWorkspace) for reason: Busy ("Application "com.pspdfkit.coretests.ios" is installing or uninstalling, and cannot be launched")., BSErrorCodeDescription=RequestDenied}
2016-12-19 11:15:17.060 xcodebuild[4519:1006396] Error Domain=IDETestOperationsObserverErrorDomain Code=5 "Early unexpected exit, operation never finished bootstrapping - no restart will be attempted" UserInfo={NSLocalizedDescription=Early unexpected exit, operation never finished bootstrapping - no restart will be attempted}

Steps to Reproduce:
Test scheme with multiple test targets.

Expected Results:
All test targets run.

Actual Results:
Some test targets run, until a test targets exits early because it encountered an error. This happens in ~10% of all cases.

Version:
macOS 10.12.2
Xcode 8.2 (8C38)

Notes:
After upgrading to Xcode 8.2 we had simulator bootup issues, but before any test targets ran. A reboot fixed these issues, but now we're occasionally seeing the same simulator bootup issues (according to the session log files) between test targets.

These bootup issues happened in earlier versions as well. We can say that the overall reliability of Xcode 8.2 is better now, this is the one major issue that remains on test stability. A re-run usually "resolves" this but we'd love to help really fixing that. Happy to provide more logs if required or better detail how our test setup looks like. We have 6 targets that run after one another per test run.

Configuration:
Mac mini 2014

Attachments:
'DTXConnection-logs.zip' and 'logs.zip' were successfully uploaded.