Open Radar

 
Gatekeeper (syspolicyd) becomes confused when an .app bundle it assessed previously changes such that its CFBundleExecutable’s name is different. spctl --assess indicates “internal error in Code Signing subsystem” as the error. Attempting to launch the .app from the UI results in this Gatekeeper dialog:

“TestApp” is damaged and can’t be opened. You should eject the disk image.
“TestApp” is on the disk image “TestApp.dmg”. Google Chrome downloaded this disk image today at 12:00 PM from example.com.
[Cancel] [Eject Disk Image]

It is very disconcerting to see that a valid signed application is considered damaged and dangerous by the system.

The simple test case involves replacing one entire application with another at the same path, having syspolicyd assess both the old and new (requested by spctl).

$ rsync -a --delete /Applications/Calculator.app/ /tmp/SysPolicyTest.app/
$ touch /tmp/SysPolicyTest.app
$ spctl --assess -vv /tmp/SysPolicyTest.app
/tmp/SysPolicyTest.app: accepted
source=Apple System
origin=Software Signing
$ rsync -a --delete /Applications/TextEdit.app/ /tmp/SysPolicyTest.app/
$ touch /tmp/SysPolicyTest.app
$ spctl --assess -vv /tmp/SysPolicyTest.app
/tmp/SysPolicyTest.app: internal error in Code Signing subsystem

The reduced test case shows that the problem is triggered when the only thing in the .app bundle that has changed is its CFBundleExecutable’s name.

$ cd testapp
$ make
cc -g -Os -c testapp.m -o testapp.o
cc -Wl,-dead_strip testapp.o -o testapp -framework AppKit
rm -f -rf TestApp.app
mkdir -p TestApp.app/Contents/MacOS
cp testapp TestApp.app/Contents/MacOS/TestApp
cp Info.plist TestApp.app/Contents
dsymutil TestApp.app/Contents/MacOS/TestApp --out TestApp.app.dSYM
strip TestApp.app/Contents/MacOS/TestApp
$ rsync -a TestApp.app/ TestApp_v1.app/
$ rsync -a TestApp.app/ TestApp_v2.app/
$ mv TestApp_v2.app/Contents/MacOS/TestApp TestApp_v2.app/Contents/MacOS/MainExecutable
$ sed -i '' -e 's/TestApp/MainExecutable/' TestApp_v2.app/Contents/Info.plist  # to edit CFBundleExecutable
$ codesign --sign 'Developer ID Application: Me' TestApp_v1.app
$ codesign --sign 'Developer ID Application: Me' TestApp_v2.app
$ rsync -a --delete TestApp_v1.app/ TestApp_test.app/
$ touch TestApp_test.app
$ spctl --assess -vv TestApp_test.app
TestApp_test.app: accepted
source=Developer ID
origin=Developer ID Application: Me
$ rsync -a --delete TestApp_v2.app/ TestApp_test.app/
$ touch TestApp_test.app
$ spctl --assess -vv TestApp_test.app
TestApp_test.app: internal error in Code Signing subsystem

You get the “internal error” even though there’s nothing wrong with TestApp_v2.app. spctl (actually syspolicyd) is fine with each app at its own path, as long as it hasn’t seen that path with a different CFBundleExecutable name previously:

$ spctl --assess -vv TestApp_v1.app 
TestApp_v1.app: accepted
source=Developer ID
origin=Developer ID Application: Me
$ spctl --assess -vv TestApp_v2.app 
TestApp_v2.app: accepted
source=Developer ID
origin=Developer ID Application: Me

spctl is also fine with it if you get it to look at the bundle itself, without consulting syspolicyd. Note that “sudo” is needed due to bug 23611423.

$ sudo spctl --assess -vv --direct TestApp_test.app
TestApp_test.app: accepted
source=Developer ID
origin=Developer ID Application: Me

This bug will occur “in the wild” as it is not unheard of for an application’s CFBundleExecutable’s name to change from one version to the next. Googling for the error “internal error in Code Signing subsystem” gives https://forums.developer.apple.com/thread/17740 as a top result. Based on that report, I downloaded http://assets.geneious.com/installers/geneious/release/Geneious_mac64_8_1_7.dmg and http://assets.geneious.com/installers/geneious/release/Geneious_mac64_9_0_4_with_jre.dmg and found that this bug was the cause of their problem too: their main executable moved from Geneious.app/Contents/MacOS/JavaApplicationStub to Geneious.app/Contents/MacOS/JavaAppLauncher.

Upon reaching this broken state, a workaround is to kill syspolicyd (sudo launchctl stop com.apple.security.syspolicy, it’ll be restarted when it’s needed again) or to reboot the computer.

Consider also that a related problem occurs if syspolicyd first assesses an .app bundle that has not been completely unpacked (for example, as it’s being copied or installed).

$ rsync -a --delete /Applications/Calculator.app/ /tmp/Calculator.app/
^Z  # interrupt it while copying
[1]+  Stopped  rsync -a --delete /Applications/Calculator.app/ /tmp/Calculator.app/
$ spctl --assess -vv /tmp/Calculator.app
/tmp/Calculator.app: invalid resource directory (directory or signature have been modified)
$ fg
rsync -a --delete /Applications/Calculator.app/ /tmp/Calculator.app/
$ touch /tmp/Calculator.app
$ spctl --assess -vv /tmp/Calculator.app
/tmp/Calculator.app: invalid resource directory (directory or signature have been modified)
$ diff -r /Applications/Calculator.app /tmp/Calculator.app && echo identical
identical
$ spctl --assess -vv /Applications/Calculator.app
/Applications/Calculator.app: accepted
source=Apple System
origin=Software Signing

Steps to Reproduce:
$ tar -jxf testapp.tar.bz2
$ cd testapp
$ make
cc -g -Os -c testapp.m -o testapp.o
cc -Wl,-dead_strip testapp.o -o testapp -framework AppKit
rm -f -rf TestApp.app
mkdir -p TestApp.app/Contents/MacOS
cp testapp TestApp.app/Contents/MacOS/TestApp
cp Info.plist TestApp.app/Contents
dsymutil TestApp.app/Contents/MacOS/TestApp --out TestApp.app.dSYM
strip TestApp.app/Contents/MacOS/TestApp
$ rsync -a TestApp.app/ TestApp_v1.app/
$ rsync -a TestApp.app/ TestApp_v2.app/
$ mv TestApp_v2.app/Contents/MacOS/TestApp TestApp_v2.app/Contents/MacOS/MainExecutable
$ sed -i '' -e 's/TestApp/MainExecutable/' TestApp_v2.app/Contents/Info.plist  # to edit CFBundleExecutable
$ codesign --sign 'Developer ID Application: Me' TestApp_v1.app
$ codesign --sign 'Developer ID Application: Me' TestApp_v2.app
$ rsync -a --delete TestApp_v1.app/ TestApp_test.app/
$ touch TestApp_test.app
$ spctl --assess -vv TestApp_test.app
TestApp_test.app: accepted
source=Developer ID
origin=Developer ID Application: Me
$ rsync -a --delete TestApp_v2.app/ TestApp_test.app/
$ touch TestApp_test.app
$ spctl --assess -vv TestApp_test.app
TestApp_test.app: internal error in Code Signing subsystem

Expected Results:
On 10.10.5 14F1021:

$ spctl --assess -vv TestApp_test.app
TestApp_test.app: accepted
source=Developer ID
origin=Developer ID Application: Me

TestApp_test.app can be launched from the UI (or by “open TestApp_test.app”) even if quarantined.

Actual Results:
On 10.11.1 15B42:

$ spctl --assess -vv TestApp_test.app
TestApp_test.app: internal error in Code Signing subsystem

If quarantined, TestApp_test.app can’t be launched from the UI (or by “open TestApp_test.app”). Instead, this dialog appears:

“TestApp” is damaged and can’t be opened. You should eject the disk image.
“TestApp” is on the disk image “TestApp.dmg”. Google Chrome downloaded this disk image today at 12:00 PM from example.com.
[Cancel] [Eject Disk Image]

Version:
This bug occurs in 10.11.1 15B42. It does not occur in 10.10.5 14F1021.

Attachments:
'testapp.tar.bz2' was successfully uploaded.