NSURLSessionStreamTask doesn't try to use HTTP Tunnelling

Originator:thibault.ml
Number:rdar://30396646 Date Originated:07-Feb-2017
Status:Closed (Dupe) Resolved:Yes
Product:macOS + SDK Product Version:10.12.3 (16D32)
Classification:Other Bug Reproducible:Always
 
Area:
Networking

Summary:
NSURLSession APIs are convenience APIs made to help the developer, and from what I can understand, should avoid the developer having to manually handle proxy settings.

NSURLSessionStreamTask was created to allow developers to use raw TCP/IP connections over URL based ones (such as HTTP). However, it seems like the API only looks for potential SOCKS proxies, and completely ignores any HTTP(S) proxy that might be set.

I find this highly surprising, as it is absolutely possible to use an HTTP(S) proxy for raw TCP/IP connections, simply by using the HTTP CONNECT method (RFC 2817 <https://tools.ietf.org/html/rfc2817>)

I believe NSURLSessionStreamTask should handle HTTP(S) proxy by using HTTP Tunnelling through the CONNECT method, without requiring developers to do it.

Steps to Reproduce:
1. Install an HTTP proxy server (for example "tinyproxy" which can be installed locally using brew)
2. Configure proxy to have it run on 127.0.0.1 and port 8888. Ensure logging is enabled
3. Configure proxy to allow connection to port 587
4. Edit /etc/hosts to have tiny.proxy point to 127.0.0.1
5. Configure system setting to have both HTTP and HTTPS proxies point to 127.0.0.1:8888
6. Ensure no SOCKS proxy are set (must be disabled)
7. On a new project (or the one attached here), create an NSURLSession with +[NSURLSessionConfiguration defaultSessionConfiguration]
8. Ensure `connectionProxyDictionary' is `nil' to request the API to use the system's settings
9. Create a stream task to a remote server (eg. smtp-relay.gmail.com on port 587)
10. Resume the task
11. Read data from SMTP server

Expected Results:
1. Little Snitch asks whether tinyproxy can access smtp-relay.gmail.com on port 587
2. tinyproxy logs show a CONNECT request to smtp-relay.gmail.com

Actual Results:
1. Little Snitch asks whether Xcode or the running binary can access theguardian.com on port 587
2. tinyproxy logs show no request to smtp-relay.gmail.com

Version:
10.12.3 (16D32)

Notes:
Setting a SOCKS proxy will actually work, in that NSURLSessionStreamTask will try to connect to it, but if none are set, it doesn't try the HTTP proxies.

Attached files:
https://www.dropbox.com/s/w9tkge4j2l47isk/NSURLSessionStreamTask_HTTPProxies.zip?dl=0

Comments

Hello Thibault,

Engineering has determined that your bug report (30396646) is a duplicate of another issue (26807970) and will be closed.

The open or closed status of the original bug report your issue was duplicated to appears in the yellow "Duplicate of XXXXXXXX" section of the bug reporter user interface. This section appears near the top of the right column's bug detail view just under the bug number, title, state, product and rank.

An example of the duplicate section from the bug reporter user interface with your bug and the duplicate bug info is included below:

30396646 NSURLSessionStreamTask doesn't try to use HTTP Tunnelling

State: Closed Product:

Rank: No Value

Duplicate of 26807970 (Open or Closed; log in to see the actual state)

By thibault.ml at Feb. 10, 2017, 11:15 a.m. (reply...)

Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!