Using FaceID to get a focus area

Originator:michaelhochs
Number:rdar://35718522 Date Originated:28-Nov-2017 10:28 AM
Status:Open Resolved:
Product:iOS + SDK Product Version:12
Classification:Feature (New) Reproducible:Always
 
Summary:
FaceID is an awesome feature and I love my new iPhone X. The most impressive thing for me (besides the fact that it actually knows how I look, of course) is the attention awareness reported by Face ID and the little tweaks that are made in the behavior of the phone with this.

What I would love to see is an API that calculates a rough area of where the user is most likely looking at on the screen. This ‘focus’ API could be used to make educated guesses of what the user is currently doing even though they are not touching the screen.

For example in PSPDFKit we use a heuristic approach to figure out which page of a PDF document the user is currently looking at to then show the user which page this is. These pages are usually displayed in aspect fit, full screen. So depending on the aspect ratio of a page there usually are no more than two or three pages visible at the screen.

A focus API that would tell us which third of the screen the user is looking at therefore would already be enough resolution to show the user the correct page number instead of just showing them all the pages currently visible. This would also help for scenarios where rotating the device changes the amount of content shown on the screen. E.g. a split view controller could decide to leave the side bar open when rotating to portrait if the user is currently looking at that.

While our heuristic approach comes close to the correct page, it fails from time to time due to its nature. A focus API would improve the correctness of this by a lot.

Steps to Reproduce:
 

Expected Results:
 

Actual Results:
 

Version:
12

Notes:

Comments


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!