[Chrome] dlopen, when given an absolute path, loads unintended dylibs

Originator:mark
Number:rdar://FB9725981 Date Originated:2021-10-27
Status:Open Resolved:
Product:macOS/‌Something else not on this list Product Version:12.0.1 21A559
Classification:Incorrect/Unexpected Behavior Reproducible:Always
 
Starting in macOS 12 (“Monterey”), when you attempt to dlopen a dylib by absolute path, if that path does not exist, dlopen attempts a “fallback” by trying to load the dylib from other locations, such as the dyld shared cache. This is contrary to the POSIX spec for dlopen, which is clear on the behavior:

https://pubs.opengroup.org/onlinepubs/9699919799/functions/dlopen.html

> The file argument is used to construct a pathname to the executable object file.
> If file contains a <slash> character, the file argument is used as the pathname
> for the file. Otherwise, file is used in an implementation-defined manner to yield
> a pathname.

macOS does make some specific extensions to this logic for paths that look like framework paths. These exceptions are not relevant to this bug or discussion.

I discovered this problem when attempting to build the Net::SSLeay Perl module for MacPorts. See https://github.com/macports/macports-ports/pull/12704 and https://trac.macports.org/ticket/63415. That module’s Makefile.PL contained logic that caused it to attempt to locate a suitable libcrypto by attempting to dlopen /opt/local/libcrypto.dylib before attempting /opt/local/lib/libcrypto.dylib. MacPorts OpenSSL does have its libcrypto.dylib available at the latter path, but there’s nothing at the former. As of macOS 12, dyld handling of the missing dylib has changed, and libcrypot.dylib is loaded from /usr/lib/libcrypto.dylib (that path is physically absent on disk, but the dylib is loaded from the dyld shared cache). libcrypto.dylib is “poisoned” to prevent being loaded by non-Apple executables, and has a load-time initializer that calls abort().

To see this in action, with MacPorts installed in /opt/local (the default), and MacPorts perl5.28 installed (sudo port install perl5.28):

mark@sweet16 zsh% curl --remote-name --silent https://www.cpan.org/modules/by-module/Net/Net-SSLeay-1.90.tar.gz
mark@sweet16 zsh% tar -zxf Net-SSLeay-1.90.tar.gz 
mark@sweet16 zsh% cd Net-SSLeay-1.90 
mark@sweet16 zsh% /opt/local/bin/perl5.28 Makefile.PL
Do you want to run external tests?
These tests *will* *fail* if you do not have network connectivity. [n] 
*** Found OpenSSL-1.1.1l installed in /opt/local
*** Be sure to use the same compiler and options to compile your OpenSSL, perl,
    and Net::SSLeay. Mixing and matching compilers is not supported.
Checking if your kit is complete...
Looks good
WARNING: /opt/local/bin/perl5.28 is loading libcrypto in an unsafe way
zsh: abort      /opt/local/bin/perl5.28 Makefile.PL

Presumably, you do not see this problem with the same Perl module using the Apple-shipped Perl, because the Apple-shipped interpreter is signed by Apple and is not forbidden to load libcrypto.dylib. For example (using Python this time), Apple code can load /usr/lib/libcrypto.dylib with just a stern warning but without a hard abort by default:

mark@sweet16 zsh% /usr/bin/python2
[…]
>>> import ctypes
>>> ctypes.cdll.LoadLibrary('/usr/lib/libcrypto.dylib')
WARNING: Executing a script that is loading libcrypto in an unsafe way. This will fail in a future version of macOS. Set the LIBRESSL_REDIRECT_STUB_ABORT=1 in the environment to force this into an error.
<CDLL '/usr/lib/libcrypto.dylib', handle fff1407e9340 at 1051e8050>
>>>

while non-Apple code cannot:

mark@sweet16 zsh% /opt/local/bin/python3
[…]
>>> import ctypes
>>> ctypes.cdll.LoadLibrary('/usr/lib/libcrypto.dylib')
WARNING: /opt/local/Library/Frameworks/Python.framework/Versions/3.9/Resources/Python.app/Contents/MacOS/Python is loading libcrypto in an unsafe way
zsh: abort      /opt/local/bin/python3

With a test program, I can easily demonstrate the problem:

mark@sweet16 zsh% clang -Wall -Werror t_dlopen.c -o t_dlopen

# OK to load my libcrypto.dylib when it’s present.
mark@sweet16 zsh% ./t_dlopen /opt/local/lib/libcrypto.dylib

# Control: forbidden to load Apple libcrypto.dylib by absolute path.
mark@sweet16 zsh% ./t_dlopen /usr/lib/libcrypto.dylib
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /usr/lib/libcrypto.dylib

# dlopen should have failed (ENOENT), but instead it loaded Apple libcrypto.dylib
# which aborts.
mark@sweet16 zsh% ./t_dlopen /opt/local/libcrypto.dylib
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /opt/local/libcrypto.dylib

# Same:
mark@sweet16 zsh% rm -f /tmp/libcrypto.dylib
mark@sweet16 zsh% ./t_dlopen /tmp/libcrypto.dylib      
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /tmp/libcrypto.dylib

# Same. This time, instead of being missing, the file exists but is not a dylib.
# dlopen should have failed, but instead it loaded Apple libcrypto.dylib which
# aborts.
mark@sweet16 zsh% touch /tmp/libcrypto.dylib
mark@sweet16 zsh% ./t_dlopen /tmp/libcrypto.dylib
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /tmp/libcrypto.dylib

# When the file is a dylib, dlopen succeeds.
mark@sweet16 zsh% cp /opt/local/lib/libz.dylib /tmp/libcrypto.dylib

# When the file can’t be opened, dlopen should have failed (EPERM), but instead it
# loads Apple libcrypto which aborts.
mark@sweet16 zsh% chmod 0 /tmp/libcrypto.dylib 
mark@sweet16 zsh% ./t_dlopen /tmp/libcrypto.dylib
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /tmp/libcrypto.dylib

# When it’s a directory instead of a file, dlopen should have failed (EISDIR), but
# instead it loads Apple libcrypto which aborts.
mark@sweet16 zsh% rm -f /tmp/libcrypto.dylib
mark@sweet16 zsh% mkdir /tmp/libcrypto.dylib
mark@sweet16 zsh% ./t_dlopen /tmp/libcrypto.dylib
WARNING: /Users/mark/t_dlopen is loading libcrypto in an unsafe way
zsh: abort      ./t_dlopen /tmp/libcrypto.dylib

I experience the above on:

mark@sweet16 zsh% sw_vers
ProductName:	macOS
ProductVersion:	12.0.1
BuildVersion:	21A559

On macOS 11.6 and earlier, I do not see the above problems. dlopen behaves according to my expectations.

Comments

t_dlopen.c

base64 --decode <<< ' H4sIAIpZeWECA2WObUvDMBSFv+dXnG0IaSmifnQvUGqFQUFYN/BlZdQkXQMxLWk7BN1/N2l1 TPblcjj3nnueidRMdVxgxlXB9HW5IJOTJYz5bzQtl9WFpeS784jULT5yqakTudmzAKzMDXyr D2+Zhy8CyALU7TCa426wgKI2NlNQ+8x2Bhh3Tb4X97hqMKvztlxs9ThA/+Ym86Z9xoi2Mxrx 83K9ewyXyWYVu8WR2NHXsko3LXyXx3wI32bu5FBJDp8rZ3NV1UJTdxRgtU4edkn4+oLvX/0U hUnf57BHNvJHbDE/6Xm3hR5+OWoLy5U9qQz1vBPVOXG6iaI4TafkSH4A8P6v8oMBAAA=' | gunzip > t_dlopen.c


Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!