Memory Allocation Fault in LU factorization of the Accelerate framework

Originator:ivan.nazarov
Number:rdar://30868533 Date Originated:2017-03-06
Status:Open Resolved:No
Product:Accelerate framework Product Version:
Classification: Reproducible:Always
 
This issue was encountered in issue 7626 in the Github repo of Scikit-Learn. In short, when trying to perform
linear dimensionality reduction for data analysis purposes, the script fails abruptly with a memory allocation error.

After some debugging the source of the fault was localized to some call within dgetrf() LAPACK routine.
The fault is reproducible with the following C++ code, which I attach to this bug report. Please, note that
changing the number of rows from 46336 to any smaller positive integer works fine.
```
#include <Accelerate/Accelerate.h>

int main (int argc, char const *argv[])
{
    // Changing 46336 to 46335 (or any lower value) works fine
    int num_rows=46336, num_cols=110;

    // create data array for our matrix. Will be a vector that is num_rows*num_cols elements
    // to access A(r,c) use A[r+c*num_rows], in major column ordering. {r1c1,r2c1,r3c1...}
    double *A = new double[num_rows*num_cols];
    for(int row=0; row<num_rows; row++) {
        for(int col=0; col<num_cols; col++) {
            A[row+col*num_rows] = 1.0;
        }
    }

    __CLPK_integer m, n, lda, info;
    m=lda=num_rows;
    n=num_cols;

    __CLPK_integer * ipiv = new __CLPK_integer[(num_rows>num_cols?num_cols:num_rows)];

    dgetrf_(&m, &n, A, &lda, ipiv, &info);

    return 0;
}
```
This is to be compiled with `g++ file.cpp -o file -framework accelerate`

The fault message reads as follows:
```
file(1490,0x7fff79ad6300) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
```
The value 18446744065123717120 is FFFF FFFE 003E 9000 in hex looks like a signed extension of an unsigned int.

Unfortunately I couldn't get a crash report from the C++ example, but I do have a crash report
from a python (2.7.13) example, which I attach to this bug report as well.

Steps to Reproduce:
1. install xcode command line tools;
2. compile the attached `file.cpp` with ` g++ file.cpp -o file -framework accelerate`;
3. execute ` ./file `.

Expected Results:
I expected a clean termination of the compiled program after completing these steps.

Actual Results:
After completing these steps got a message in the terminal, which looked like the one below:
```
file(1490,0x7fff79ad6300) malloc: *** mach_vm_map(size=18446744065123717120) failed (error code=3)
```

Version:
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

System Version: OS X 10.10.5 (14F2109)
Kernel Version: Darwin 14.5.0

Update 2017-03-07:
This fault is reproducible on the following versions of OS X:
```
      System Version: OS X 10.9.5 (13F1911)
      Kernel Version: Darwin 13.4.0
```

```
      System Version: OS X 10.11.5 (15F34)
      Kernel Version: Darwin 15.5.0
```

```
      System Version: OS X 10.11.6 (15G1217)
      Kernel Version: Darwin 15.6.0
```

Comments

Update 2017-08-30

On the latest Sierra `System Version: macOS 10.12.6 (16G29) Kernel Version: Darwin 16.7.0 ` I get the segfault for these constants: `int num_rows=46344, num_cols=110; ` Changing 46343 to 46344 or any higher value causes a segfault.

By ivan.nazarov at Aug. 30, 2017, 8:50 a.m. (reply...)

Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!