AVFoundation face yawAngle metadata broken on portrait iPhone 8 Plus
| Originator: | alexisgallagher | ||
| Number: | rdar://35064476 | Date Originated: | 2017-10-18 |
| Status: | Open | Resolved: | |
| Product: | iOS/AVFoundation | Product Version: | iOS 11.0.3, iPhone 8Plus |
| Classification: | serious bug | Reproducible: | always |
Area: AVFoundation Summary: If you configure AVFoundation to detect faces on an iPhone 8 Plus and also hold the device in portrait mode, AVFoundation does not report the face yawAngle correctly. Steps to Reproduce: Get an iPhone 8 Plus, hold it upright in portrait mode, and run the enclosed sample project. This project contains my minimal modifications to Apple sample code for the AVFoundation face detector. One modification is just to add an overlay layer that displayed the detected face yawAngle onscreen. The other, critical modification is to configure AVFoundation to detect *only faces*, rather than all possible metadata types. This last change triggers the bug. While still holding the device in portrait mode, turn your head to left and right. Expected Results: I expect to see the yawAngle displayed — - as it is on any device besides an iPhone 8 Plus (or iPhone 8) - as it is if I physically re-orient the iPhone 8 Plus so it's screen faces the sky, and then bring it back to portrait orientation. - as it is if I, while holding the device in portrait mode, roll my head 90 degrees counterclockwise around the axis going from my nose to the phone, and _then_ turn my head left or right. (In other words, the faceYaw variable is behaving as if it thinks the device is in landscape orientation, and this weird motion lines my head up with that expectation) Actual Results: I do not see yawAngle updated at all. Version/Build: 11.0.3 (15A432)
Comments
Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!
Restarting the device makes the fix work again
New information, as of 2017-10-24T14:34 SF time, regarding what is required in order to make my "all possible metadata" fix work. If you restart the device, then the fix works. So to recap, here is what we've observed:
If you:
If you: - are running an iPhone 8 Plus (on iOS 11.0.3) - hold it in portrait orientation for the entire duration of this test - configure AVFoundation to detect all metadata then: AVFoundation MIGHT correctly detect face yawAngle
If you: - are running an iPhone 8 Plus (on iOS 11.0.3) - hold it in portrait orientation for the entire duration of this test - configure AVFoundation to detect all metadata - restart the device before running this test then: AVFoundation WILL correctly detect face yawAngle
I realize this sounds crazy. But we have been careful with our observations here, since this is a critical component for our app.
The fix has now stopped working (I don't know why)
As of 2017-10-24T13:51 SF time, this fix has stopped working for me but the bug persists. I cannot imagine why this is. My only guess is that maybe the fix worked on 11.0.2 but does not work on 11.0.3, and I recorded the Version/Build incorrectly. But I think this is unlikely.