Open Radar

 
Area:
Something not on this list

Summary: Calling readdir() from multiple threads apparently acquires a global kernel lock, making directory traversal operations from multiple processes extremely slow as the number of parallel I/O operations increases.


Steps to Reproduce:

I have a Gist at https://gist.github.com/indygreg/a50e187f5372807cdcab5ac12bc2feea that demonstrates the issue using Python.


Expected Results:

It would be nice if read-only parallel I/O scaled linearly (within reason). Other filesystems (like EXT4) don't exhibit excessive kernel CPU time performing the same type of I/O operations in parallel.

In addition, it would be useful if the APFS developer documentation documented which I/O operations are subject to global locks so developers know how to optimize parallel I/O under APFS.

Actual Results:

Parallel I/O performing readdir() results in excessive CPU time being spent in the kernel acquiring locks.

Version/Build:

I've reproduced on macOS 10.13.6 and 10.14 on a MacBook Pro 15.1. Others have reproduced on other devices running macOS. I assume the issue is intrinsic to APFS.

Performance on 10.14 is noticeably better than 10.13. But performance is still slower lagging, especially when compared to Linux/EXT4.

Configuration:

Nothing special. Was able to reproduce on a fresh 2018 MacBook Pro straight from Apple.