Open Radar

 
Please describe the issue:

The iOS / tvOS project I am working on could always be successfully archived on our continuous integration / delivery servers running Xcode 12.x, both for internal nightly / beta distribution as well as for TestFlight and App Store Connect. These servers have run all versions of Xcode 12.x and now run 12.5. After our servers were updated to Xcode 13 RC, though, archiving of our application started to fail almost all the time.

By looking at the compilation logs we discovered that this archiving problem occurs when compiling one of our SPM source packages A, which itself uses a binary SPM package B as transitive dependency. The line on which compilation fails is a line in the source code of A where the `B.framework` is imported, with the error message "Module 'B' not found", indicating that Xcode failed to locate B.framework. I first suspected `B.XCFramework` might not be available from the local SPM cache for some reason but after verification everything looked normal.

To investigate this issue further I started from a simple project (Xcode iOS app template), just adding A as SPM dependency (and thus transitively B). To my surprise this simple project could successfully be archived, and reliably. I then started adding other SPM dependencies from GitHub to this simple project until I found that, when a sufficiently large number of dependencies have been added (~6 additional packages in my case), archiving starts to fail.

The difference can be seen in the compilation logs. When everything works fine (Xcode 12.5 or small number of SPM dependencies), the `ProcessXCFramework` operation is performed early in the compilation cycle for each involved SPM binary package, making the binary available to compile packages depending on it. This is what ensures that B.framework is available when the A package is compiled. When too many SPM dependencies are added to the project, though, the `ProcessXCFramework` step does not appear in the log anymore for some obscure reason, which is why compilation of the A package fails.

We also learned a few things during our investigations:

- When using Build & run the problem does not seem to happen (this is why we discovered it on our CI servers first, though archiving on a development Mac suffers from the same issues).
- When enabling "Manual order" instead of "Dependency order" in the application scheme build settings, the problem disappears entirely. If toggling the setting back to "Dependency order" the problem appears again. I therefore suspect that some kind of racing condition in build parallelization could explain this issue.
- Similar issues have been reported on the Swift forums (e.g. https://forums.swift.org/t/use-xcodebuild-swift-package-with-binarytarget-logic-flaw/41579) and might be related.

Thank you very much for investigating this issue.



Please list the steps you took to reproduce the issue:

I joined a sample project to this issue with which your can reproduce the issue. Since we might be facing a race condition here are our hardware specifications:
 - MacBook Pro (15-inch, 2016), 2.7 GHz Quad-Core Intel Core i7.
 - Mac Mini, 3.0GHz Intel Core i5 6-Core (we have 3 of them for our CI).

Please follow the steps below to reproduce the issue:

1. Open the attached zip file which contains a sample project with Xcode 13 RC.
2. This project itself is a blank app with almost no code, but a few dependencies:
    - SRG Analytics is an SPM package we offer, which uses other binary packages (comScore, TCCore, TCSDK) as dependencies. What the code does is irrelevant, these packages simply play the role of A and B from the discussion above, respectively. You can check the SRG Analytics associated Package.swift to see how the binary packages are referenced (the package manifest is also attached to this issue; these packages are publicly available as well on GitHub).
    - A mixture of other 3rd party dependencies pulled with SPM. 
3. Provide a code signing team and an app id so that code signing is setup properly for archiving.
4. Archive the project with Xcode 13 RC. Archiving fails with the "Module 'xxx' not found" error (xxx is either comScore, TCCore or TCSDK, one of our binary dependencies).

If you perform the same steps with Xcode 12.5 archiving is successful in all cases.

Using Xcode 13 RC you can also verify the following observations and maybe gain some additional insights:

- Building and running the project in the simulator or on a device works fine.
- If you enable "Manual order" in the application scheme build settings the project archives fine.
- If you restore "Dependency order" in the scheme build settings and remove a few SPM dependencies (keeping e.g. only SRG Analytics and swift-collections), the project archives fine.

I also attached two compilation logs to this issue:

- `Archive_failure.txt`: Obtained with the sample project setup. You can verify that `ProcessXCFramework` is nowhere to be found in the log.
- `Archive_success.txt`: Obtained with the sample project by enabling "Manual order" in the scheme build settings, which makes archiving successful. `ProcessXCFramework` can be found at the top of the log.



What did you expect to happen?

Archiving a project having any number of dependencies (source and binary packages, some of them possibly with transitive dependencies) should work, as it was the case with Xcode 12.5.



What actually happened?

Archiving a project having several dependencies and binary transitive dependencies fails in Xcode 13 RC. As the number of packages is increased packages referencing binary packages will fail to compile correctly.