-[XCTestCase expectationWithDescription:] takes 70+ seconds to create an expectation

Originator:B.Gesiak
Number:rdar://20574502 Date Originated:16-Apr-2015 03:08 PM
Status:Open Resolved:
Product:Developer Tools Product Version:
Classification:Serious Bug Reproducible:Always
 
Summary:
In some projects, -[XCTestCase expectationWithDescription:] takes over 70 seconds to create an XCTestCaseExpectation instance.

The following test case file reproduces the problem:

```
#import <XCTest/XCTest.h>

@interface LoweredExpectationsTests : XCTestCase
@end

@implementation LoweredExpectationsTests

- (void)testExample {
  XCTestExpectation *expectation = [self expectationWithDescription:@"some random expectation"];
  [expectation fulfill];
  [self waitForExpectationsWithTimeout:1.0 handler:nil];
}

@end
```

This test case, when run on a new Xcode project with very little code in it, executes in around 0.007 seconds. However, in a large project, the same test case takes upwards of 70 seconds--76.767 seconds, to be exact.

Because the project that consistently reproduces this error is proprietary, I cannot send a sample project. I have, however attached a sysdiagnose report. To product the report, I:

1. Ran sysdiagnose, passing the pid of Xcode.
2. While that was running, I ran the unit tests in Xcode for the project that reproduces the issue.
3. Waited until sysdiagnose completed.

Please let me know if there's a better way to report information on this error.

To diagnose, I paused execution of the test using the debugger while it was taking 70+ seconds to execute -[XCTestCase expectationWithDescription:]. I've attached a text file containing the stack trace (also available online at https://gist.github.com/modocache/da56bff7703682f6311d). My theory is that my library contains a lot of code, and CoreSymbolication is having a hard time symbolicating. I'm not sure why symbolication is occurring.

Steps to Reproduce:
1. Add the following test case to a large codebase:

```
#import <XCTest/XCTest.h>

@interface LoweredExpectationsTests : XCTestCase
@end

@implementation LoweredExpectationsTests

- (void)testExample {
  XCTestExpectation *expectation = [self expectationWithDescription:@"some random expectation"];
  [expectation fulfill];
  [self waitForExpectationsWithTimeout:1.0 handler:nil];
}

@end
```

2. Run the test case.

Expected Results:
The test case executes and passes immediately.

Actual Results:
The test case takes a very long time to run, in my case over 70 seconds.

Version:
Version 6.3 (6D570), OS X 10.10.2 (14C1514)

Notes:


Configuration:


Attachments:
'sysdiagnose_2015.04.16_14-30-14-0400_37345.tar.gz' and 'xctestcase-stack-trace.txt' were successfully uploaded.

Comments

Brian Gesiak 16-Apr-2015 04:52 PM

I ran another sysdiagnose, this time on the xctest process itself (the previous one was on the Xcode.app process), and attached the file. Steps I took:

  1. Add a unit test to a test target for a large Xcode project. The test method is as follows:

` @interface FBGraphQLConnectionStorePersistentPageLoaderTests : XCTestCase @end

@implementation FBGraphQLConnectionStorePersistentPageLoaderTests

// ...

  • (void)test_loweredExpectations { XCTestExpectation *expectation = [self expectationWithDescription:@"lowered expectation"]; [expectation fulfill]; [self waitForExpectationsWithTimeout:1.0 handler:nil]; }

@end `

  1. Insert a breakpoint at the top of that method, before -[XCTestCase expectationWithDescription:] is called.
  2. Run only that single unit test.
  3. When the breakpoint is hit, run sysdiagnose against the xctest process.
  4. Continue the test process.

Hopefully this dump has more relevant information for you all! 'sysdiagnose_2015.04.16_16-43-10-0400_16235.tar.gz' was successfully uploaded.


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!