NSOpenGLView content aspect ratio broken on 10.14 with VLC 4

Number:rdar://45895864 Date Originated:November 8 2018, 2:24 AM
Status:Open Resolved:
Product:macOS + SDK (AppKit) Product Version:10.14.2 Beta (18C31g)
Classification:Bug Reproducible:Always
In VLC we use a subclass of NSOpenGLView to display the video on screen.
As soon as we opt-in to 10.14's new dark mode either with NSRequiresAquaSystemAppearance in our Info.plist or by building against the 10.14 SDK the video output will not properly preserve aspect ration in the video output.

Steps to Reproduce:
1. Run a VLC nightly of VLC on 10.14:


2. Play a video and switch to fullscreen

Expected Results:
The aspect ration should be preserved when changing to fullscreen.

Actual Results:
The video ends up being distorted to fit the full screen.

10.14.2 Beta (18C31g)

I am unsure if this is some issue in VLCs subclass of the NSOpenGLView or if this is caused by some regression in the NSOpenGLView. I am interested in fixing this, not necessarily requiring a fix in macOS, but some hints what might be the bug in our code, if this is not a macOS issue. The OpenGL video output works fine on other OSes and the macOS specific variant does not seem to do anything much different that could cause this, especially since it only happens when opting in to dark mode (which afaik opts the app into full layer backing).

Any hint what the issue is would be highly appreciated. Note that this same issue does not happen with our CAOpenGLLayer video output but it's not in good shape to enable that in official releases.

The relevant code for the macOS NSOpenGLView based video output can be found here: http://git.videolan.org/?p=vlc.git;a=blob;f=modules/video_output/macosx.m;h=948e56d400e8c8a0ef8f40635fb3c48ab190048b;hb=HEAD


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!