ReplayKit, Broadcast Upload Extension: Foreground Privileges Needed for AVAssetWriter?

Originator:o.schnurr
Number:rdar://33811226 Date Originated:8/9/2017
Status:closed Resolved:yes
Product:iOS Product Version:iOS 11 beta 5 (15A5341f)
Classification:ReplayKit Reproducible:Always
 
Area:
ReplayKit

Summary:
In beta 4 of the iOS 11 SDK, I am able to create and start an AVAssetWriter successfully in a broadcast upload extension (NSExtensionPointIdentifier: com.apple.broadcast-services-upload). With this, I was able to take audio and video sample buffers that were received by the extension and write them into a video file.

In beta 5 of the iOS 11 SDK, the AVAssetWriter fails when startWriting() is called. From what I can tell from console logs, it appears that CMSessionMgr is now running the extension with "Background Running". In beta 4, it the extension was run with "Foreground Running".

(This difference between background vs foreground is my guess as to why the AVAssetWriter no longer works in a broadcast upload extension in beta 5.)

Without AVAssetWriter, broadcast upload extensions are effectively relegated to only do live broadcast. An entire category of 3rd-party upload extensions that could potentially offer some kind of interesting take on screen _recording_ are precluded.

Steps to Reproduce:
- Run the attached AssetWriterTest project to install the AssetWriterTest app and associated "Broadcast" extension on an iOS 11 beta device (beta 5 or later).
- Install the Screen Recording button into your Control Center if it's not already installed.
- Bring up the Control Center.
- 3D touch the Screen Recording button.
- Choose the "Broadcast" extension.
- Leave your device attached to a Mac and open the Console app and select your device.
- Filter the Console app down to "Errors and Faults"
- Clear the log.
- Tap "Start Broadcast".
- Wait for the countdown. Let the recording occur for at least a second or so.
- Stop the recording

Expected Results:
You _don't_ see the following error in the Console log: Broadcast "startWriting() failure: The operation could not be completed".

Observed Results:
You _do_ see the following error in the Console log: Broadcast "startWriting() failure: The operation could not be completed"

Version:
- iOS 11 beta 5 (15A5341f)
- Xcode 9 beta 5 (9M202q)

Notes:
note: An equivalent test result can be achieved by attaching the Xcode debugger:

- In Xcode select Debug -> Attach to Process by PID or Name ... -> Broadcast.
- Set a break point inside the _testAssetWriter() method in SampleHandler.swift before doing the recording on the device.
- Step through the code when the break point is trapped.

The extensions effected by the bug in the sysdiagnose are com.TechSmith.Capture.Recorder and com.TechSmith.Labs.AssetWriterTest.Broadcast. Last reproduced around 5pm eastern time on Aug 9, 2017.

Configuration:
Only a beta 5 device causes this problem (regardless of whether the test app is compiled with beta 4 or 5 of Xcode 9).

Comments

Resolved by Apple in iOS 11 beta 7 on Aug 21, 2017. The broadcast upload extension appears to run with foreground privileges again.


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!