Xcode is unusably slow during indexing

Originator:steipete
Number:rdar://27844227 Date Originated:15-Aug-2016 11:15 AM
Status:Open Resolved:
Product:Developer Tools Product Version:8b5
Classification:Performance Reproducible:Always
 
Peter Steinberger15-Aug-2016 11:15 AM

Summary:
We work on a large project with many sub-projects and targets, mixed Objective-C(++) and C/C++ (and very very little Swift).

While Xcode is indexing, typing often has a delay of multiple seconds, making it very sluggish and hard to use. This eventually stops once all files are indexed, but if I touch any headers the game starts from the beginning.

This is especially problematic because indexing can take 20 or more minutes on a top-of-the line rMBP. I'm basically typing blind and at some point text appears.

I don't mind if indexing takes a long time, but it really should be happening in the background. I took a sample while I was waiting for typing to appear:

https://gist.github.com/steipete/09ed94e78f084804a48291bef6c965c5

What is Xcode doing on the main thread while I type?
(I did not spend extended time on this and it's quite hard to correctly understand code-flow without having access to the source code, so some of these might be uninformed wrong guesses)

- Running a quite complex and slow regular expression via `[IDEMediaResource extractNonVariantNameFromString:withFoldingStrategy:returningVariant:]`
- Rebuilding much of the UI and running that through lots of slow AutoLayout calls (e.g. [IDEEditorContext _rebuildRightControlGroup] or _willMeasureMinSizeForFullscreen, originating from [NSView _layoutAtWindowLevelIfNeeded] - is the whole window layout invalidated each time i type?)
- Deallocating large-ish images via NSCache (-[IDEEditorHistoryItem setCachedImage:])
- Loads of extremely deeply nested [NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] calls that end up in many slow drawRect: calls.
- IDEPathCell seems to copy images within drawRect which calls to NSImageURLReferencingRepProvider (slow)
- Heavy KVO usage which is rather slow due to the nature of how things work (NSKeyValueUnnestedProperty, loads of string comparisons)
- -[IDEUtilityArea _rebuildCategoriesAndStack] seems to be expensive.

BUT. That's all acceptable and not a big deal, and much of it are small improvements and much of it is also just how AppKit works. I spun up Instruments to see what really is happening here.

https://twitter.com/steipete/status/765106084755415040

I did an instrumentation run of roughly a minute (attached as part of this radar). BY FAR the most time is spent in 3 functions, which all seem like perfect optimization candidates.

Weight	Self Weight		Symbol Name
18.27 s   31.0%	17.00 ms	 	                            -[DVTiPhoneSimulator deviceSpecificOverridingPropertiesForBuildable:withBaselineParameters:]
14.16 s   24.0%	10.00 ms	 	                            -[DVTiPhoneSimulator supportedSDKsForBuildable:buildParameters:error:]
11.48 s   19.4%	10.00 ms	 	                            -[DVTAbstractiOSDevice supportedArchitecturesForBuildable:buildParameters:error:]

It seems our main performance issue here does not originate from DevToolsCore/IDEFoundation, but IDEiOSSupportCore:

Digging deeper, here's what shows up on top of time profiling. These might be candidates for caching:

866.00 ms    1.4%	11.00 ms	 	                             -[SimDevice deviceType]
540.00 ms    0.9%	1.00 ms	 	                             -[SimDeviceType(DVTAdditions) dvt_supportedArchs]

This seems to be called a lot while I type - I do not change the configuration while I enter text, this is a likely candidate for caching:

2.73 s    4.6%	5.00 ms	 	                              -[Xcode3Target baseSDKForBuildConfigurationName:]

Especially because half of the time is spent here:
901.00 ms    1.5%	8.00 ms	 	                                +[Xcode3SDK sdkForNameOrPath:]

[DVTMacroDefinitionTable setLiteralValue:forMacroName:conditionSet:] spends almost all time on checking something confirms to a protocol - maybe this check could be moved somewhere higher up?

Weight	Self Weight		Symbol Name
14.16 s   24.0%	10.00 ms	 	                            -[DVTiPhoneSimulator supportedSDKsForBuildable:buildParameters:error:]

Again spending a ton of time in sdkForNameOrPath:

Weight	Self Weight		Symbol Name
11.48 s   19.4%	10.00 ms	 	                            -[DVTAbstractiOSDevice supportedArchitecturesForBuildable:buildParameters:error:]

Cache candidates?
809.00 ms    1.3%	5.00 ms	 	                             -[DVTiPhoneSimulator nativeArchitecture]
1.11 s    1.8%	9.00 ms	 	                              -[DVTiPhoneSimulator operatingSystemVersion]
336.00 ms    0.5%	7.00 ms	 	                              -[DVTPlatformFamily platforms]

Buidling an error string that is taking up 3.5% of the time alone. (Never seen that - could be optimized/delayed to create error description lazily?)
2.09 s    3.5%	9.00 ms	 	                              +[NSError(DVTDeviceIneligibility) dvt_errorWithDeviceIneligibilityErrorCode:device:buildable:buildParameters:]


I've only spent an hour on this radar, as I'm sure Apple has smarter people who have a better idea where to work on. This radar is even more so here to show that there is a problem, and it seems a few weeks on it could easily create a 10x or more performance win here. Xcode being sluggish ultimately makes it harder to build great apps for the iOS platform, compared to other IDEs such as Visual Studio or even Android Studio, which is much faster despite running inside a JVM.

Despite all that critique, I have to say that Xcode improved a lot over the years. I started with Xcode 3 and it's amazing how far it came. I understand that there are deadline and marketing wants even more features, and the Xcode team is doing amazing work in pushing forward fast with a relatively small sized team. You have my respect for that.

Steps to Reproduce:
1. Open a large iOS project in Xcode (maybe Pages.app?)
2. Delete DerivedData
3. Watch Xcode re-indexing the project and start typing, modify source and header files
4. Observe multi-second delays while typing.

Expected Results:
Indexing can be slow; but typing should be fast and have a < 250ms delay to be somewhat usable.

Actual Results:
Delays of 3000ms and more while typing make it really hard to work.

Version:
Xcode 8b5 (8S193k)

Notes:


Configuration:


Attachments:
'Xcode-Indexing-While-I-Type-Instruments.trace.zip' was successfully uploaded.

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!