Open Radar

 
Summary:
We think that overall file transfer speed of iCloud Drive should be urgently improved. For example, uploading of 2000 files (avg. file size 24 KiB, total file size 46 MiB) can take sometimes up to 2 hours and has an average time of 30 minutes. On the other hand, downloading the same amount of data on an iOS device takes sometimes even days and has an average time of several hours. 

Even worse, initial access to container metadata often takes sometimes up to 5 minutes. Also metadata queries behave very unpredictable - preventing any useful approach of lazy file downloads.

In our test setting we had a 5 MBit/s upstream and 100 MBit/s downstream. So even in worst case upload should be performed in less than 10 minutes. Download should be performed in less than 2-3 minutes. 

The used test setting is quite realistic: Our app Ulysses is designed to provide people access to their whole writings. Typically, an iCloud container consists of up to 3000 single files with an overall size of up to 50 MiB and a average file size of 20-100 KiB. 

We consider the full upload scenario as important since people may switch to iCloud at any time and then need to migrate their full work to iCloud fast and reliably. We also consider the full download scenario as realistic, since it may occur after every install of an app on a certain device. People should not wait hours or even days for iCloud to complete downloads and uploads.

Lazy downloading of files would be an option: However, since even the initial metadata query sometimes may even take up to 5 minutes, it is hard for us to present all ubiquitous, downloadable files to the user. Even worse: our app can't tell the user whether the presented file overview is complete or incomplete.

We also think that lazy downloading should be only needed in certain circumstances (bad network conditions). If the app operates in foreground and inside a WiFi network (even with a Mac having all the originals nearby), synchronization performance should be very high.

Steps to Reproduce:
We've created a small example project exploiting this behavior. It consists of a Mac and an iOS target:

- The Mac target creates 2000 files (16 KiB) each inside an iCloud container. It also writes stats about the upload progress using a metadata query.

- The iOS target starts a metadata query and downloads automatically all missing files. Since we've figured out that requesting all files at once may overload the system, the app makes sure not to download more than 10 files at once.

To run this example project, you probably need to setup an iCloud container and provision profile. The current settings are set to our internal requirements. Both projects write download / upload stats to the console.

Please run the example multiple times. We've experienced very strong deviations between individual runs.

Expected Results:
With 5 MBit/s upstream and 100 MBit/s downstream to a very common German cable-ISP (Kabel Deutschland):

- Upload of 46 MiB should not take more than 5-10 minutes.
- Download of 46 MiB should not take more than 2 minutes.
- Metadata should be available at least during 30 seconds. (At least change date, available files and ubiquity states. Other metadata may be available later)

Actual Results:
See the attached .txt files. 

- The file "mac-upload.txt" shows the uploading times. As you can see upload completion was notified after 36 minutes. (Interestingly, the last update stating that all files have been uploaded was never received)

- The file "ipad-download.txt" shows the downloading times. As you can see initial metadata query took 34 seconds, only delivering 20 metadata items. Until this time, the app wouldn't be able to give the user any useful information about ongoing downloads! Full metadata were available after 100s. 

After 15 Minutes less than 1/4 of files (11 MiB) have been downloaded. We stopped our benchmark then.

As you can also see, several single files took longer than 60 seconds, even though such files had a size of 24 KiB (see the "Canceled waiting" outputs)

- In "iOS-greedy-download.txt" you see another attempt where we requested all files as soon as we're knowing about them. As you can see first metadata (only 10 items!) were received after 70 seconds. The query had access to all results after 18 minutes. After 22 minutes all files have been downloaded.

- In both test runs, download of metadata didn't start immediately. Instead we had to restart the app until the metadata query provided any useful updates.

Version:
iOS 8 GM
OS X Yosemite DP8

Configuration:
MacBook Pro Retina mid-2014
iPad Air (1st Generation)