Open Radar

 
We learned through FB7665551 that Apple introduced a ContentFilterExclusionList in its NetworkExtensions framework, allowing a list of specific Apple processes/apps to bypass oversight and control from apps using NEFilterDataProvider. What is not possible anymore:
1. The identification of their network activity
2. The measurement of the volume of network traffic from those apps
3. The identification of IP adresses or domains where this traffic flows to or from
This is a regression in functionality, that was previously possible with Network Kernel Extensions.

Steps to reproduce it: 
1. Download the TripMode 3 trial from tripmode.ch
2. Install it along with its Network Extension
3. Turn the TripMode switch ON (= only allowed Apps can access the Internet)
4. Start a FaceTime call or System Update download

Expected result:
FaceTime or System Update traffic to be identified in the list of blocked apps. FaceTime call or System Update do not proceed to download/upload any data. From a developer perspective, we would expect to 
1. Be able to measure the volume of traffic on the Mac without exception, and report it to the user
2. Be able to attribute the volume of traffic to specific apps, and report it to the user
3. Allow the user to take block/allow decision on any system traffic - especially anything that has a significant data download/upload potential.

Actual result:
FaceTime or System Update are not identified in the list, their volume of traffic is not counted, and the call and system update download take place. Apple ignores the user wish to control its system Internet traffic.

Related issues: FB7665551, FB7554920, FB7426779

Comments :
The direct impact this is starting to have on our user base, who uses our app to 
1. Stop all unwanted downloads while on expensive networks (e.g. hotspots, roaming, limited data packages, throttled plans) 
2. Have control over their data: who gets to use it and when. We cannot deliver on the promise that user’s data belongs to them, and no one else, when Apple hides traffic from its own apps.

We just released TripMode 3 that uses NEFilterDataProvider and people are starting to report these issues, we’ve been trying for more than a year to solve. We investigated many solutions and alternatives for more than a year:

Rationale to decide on using NEFilterDataProvider. 
1. It's the most direction translation of the KPI we've been using to do user-space filtering.
2. It covers exactly our needs since 10.15.4 added traffic reporting, minus the whitelisting discussed here
3. High performance as we interact the less with the kernel and data to filter because decision is handled at the whole flow level.
4. Request URLs provided for webkit browsers and some system services without having to resort to packet inspection. 
5. The peek function enables some DNS replies parsing for non-TLS DNS without resorting to proxying, which simplifies reverse DNS for non-webkit browsers.

Rationale against Transparent Proxy
1. Very strange user experience from the System that our beta users don't understand in initial tests. Some of them have a connection timer for VPNs which also shows in the menu bar when Transparent Proxy is active making for legitimate questioning about what this does and what happens to their data. Some remove the proxy connection because they don't trust this entry due to preference panel wording, leading to loss of functionality.
2. Using Transparent Proxy for system-wide filtering has average to poor performance CPU and network-speed wise in our testing due to kernel callout to user-space. We understand this will eventually be mitigated in a future macOS release with possible user-space networking, but this leaves us with a bad energy footprint we've never had in the past due to efficient KPI usage. As many of our users are laptop users, this is a crucial point. 
3. Random and untraceable Kernel Panics on many of our development machines (even after clean installs and no third party KEXTs) when using Transparent proxy to grab the whole network traffic instead of the more restricted subset used in sample code. 
4. The whole thing is overkill for a simple block / allows decision + accounting which is perfectly handled by NEFilterDataProvider with as little interaction with user data as possible. 

Rationale against NEPacketFilterProvider.
1. No application context for the handler, which is a showstopper as this prevents any per-application features which is at the very base of our app. Deep packet inspection can only get a few hints but no definitive attribution.
2. Same kind of bad performance CPU and networking-wise as Transparent Proxy. 

Rationale against DNS proxy.
1. Works well for per-app domain blocking, but of course won't provide any accounting so it has to be coupled with another solution.
2. Caching in web browsers (mostly) means that there's no real-time effect when changing the settings from allow to block.
3. Straight bypass by some applications prevents filtering