Xcode/Command Line Tools: /usr/bin/python3 cannot find default SSL trust store, resulting in CERTIFICATE_VERIFY_FAILED for all HTTPS requests

Originator:i
Number:rdar://7111585 Date Originated:August 24, 2019
Status:Open Resolved:No
Product:"Something else not on the list" Product Version:
Classification:Incorrect/Unexpected Behavior Reproducible:
 
Xcode Version 11.0 beta 6 (11M392q), Command Line Tools for Xcode 11.0 beta 6 (11.0.0.0.1.1565314599). xcode-select either /Applications/Xcode.app (the beta version) or /Library/Developer/CommandLineTools.

As described in the title, all HTTPS requests using /usr/bin/python3 fail with CERTIFICATE_VERIFY_FAILED:

    $ /usr/bin/python3 -c 'import urllib.request; urllib.request.urlopen("https://www.google.com/")'
    ...
    urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)>

This is because the ssl module fails to correctly locate the default trust store in /etc/ssl:

    $ python3 -c 'import ssl; print(ssl.get_default_verify_paths())'
    DefaultVerifyPaths(cafile=None, capath=None, openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/certs')

As we can see here, it is looking inside /Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.7/etc/ssl/, which doesn't exist, causing all SSL cert verifications to fail.

Comments

Won’t fix

According to Apple it’s a “Won’t fix”. From https://github.com/HandBrake/HandBrake/issues/2216#issuecomment-527114519

So the solution is to use Requests instead.

By franklinyzlyzl at Feb. 21, 2020, 8:46 a.m. (reply...)

Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!