Updated: July 19, 2016

With the release of R 3.3.0 and Rtools34, these instructions needed yet another update. The most important difference is that Rtools34 is based on gcc 4.9, and can be used to compile OpenBLAS as well as R. MSYS2 is still necessary, but there is no more need for a separate MinGW64 GCC installation.

Compiling OpenBLAS

1. Download OpenBLAS (hereafter referred to as OPB) source code. The version used for this example is 0.2.18.
2. Download a 64-bit MSYS environment. I used 64-bit MSYS2 date-stamped 201602605.
3. Install MSYS2.
   • Be careful to follow the directions in Section III of the installation instructions as there are some unique quirks to an installation. Specifically, the installed pacman (package manager) is 5.0.0.6348, which means that the update-core method must be used. This also may require multiple forced closings and restarts of MSYS. The batch files used to start MSYS may also change. What fun!
   • Once the instructions in Section III are completed, also install make using pacman -S make from within the MSYS2 shell.
   • Once that is complete, edit the msys2.ini and mingw64.ini files to uncomment the line MSYS2_PATH_TYPE=inherit in both.
   • For some reason, there is a conflict between the key authentication of the MSYS2 package manager and GnupG4win, so you will have to resort to uninstalling GnuPG2 and sticking with GnuPG1 for the time being.
   • Once MSYS2 is installed, there will be a directory called home and underneath it one that is probably your log-in name. This second subdirectory is your actual “home”. Create a subdirectory underneath here. This will be referred to as OPB_HOME.
4. Expand/unzip the OpenBLAS source code into OPB_HOME.
5. Edit your path to remove the reference to “C:\Rtools\bin” and replace it with “X\msys64\usr\bin;“, substituting the appropriate drive letters and subdirectories as necessary X.
6. Open a command shell and type PATH ensuring that the MSYS2 paths is there and the Rtools\bin path is not. In my experience, not calling PATH runs the risk of the change not “registering” somehow.
7. Open the file Makefile.rule in the OPB source code directory with a text editor (I use Notepad++) and make the follow changes. Some of them require uncommenting, which means removing the # at the beginning of the line:
   1. Set FC = gfortran (Line 25 in v0.2.18)
   2. Set BINARY=64 (Line 46 in v0.2.18)
   3. Set USE_THREAD = 1 (Line 52 in v0.2.18). Personal experimentation indicates that in most cases multi-thread outperforms single-threaded (eigen on non-symmetric matrices is an exception).
   4. Set NUM_THREADS = 4 (Line 60 in v0.2.18). Personal experimentation indicates that setting the number of threads equal to half the number of available threads—equal to the number of physical processors factoring in hyperthreading—leads to the best results.
   5. Set NO_SHARED = 1 (Line 66 in v0.2.18)
   6. Set NO_CBLAS = 1 (Line 69 in v0.2.18)
   7. Set NO_LAPACK = 1 (Line 77 in v0.2.13)
   8. Set and comment out # NO_WARMUP = 0 (Line 96 in v0.2.18)
      • You want OpenBLAS to find more optimal settings!
   9. Set NO_PARALLEL_MAKE = 1 (Line 112 in v0.2.18). In my experience the GCC-based compilers for Windows do not handle parallel makes well.
   10. Set GEMM_MULTITHREAD_THRESHOLD = 4 (Line 145 in v0.2.18)
       • This speeds up computation by using the single thread version for small matrices where the multithread overhead outweighs the speed gain.
8. Run msys2.exe file to open an instance of MSYS2.
9. Navigate to the directory in which the source code and makefiles sit, and type “make“
10. After a while and lots of fast-scrolling text, the compilation should finish successfully. If it does not, and you followed all of the above steps, you’ll probably have a few tens of hours of online searching to do to try and figure out what went wrong, I’m afraid.

At this point, in the OPB folder within the MSYS install, there will be a file with the name libopenblas_sandybridgep-r0.2.18.a where sandybridge may be something different depending on your processor (Nehalem, Prescott, etc.). This is the compiled static BLAS which will be fed into the R compilation.

Compiling R

I am not going to go through an exhaustive step-by-step explanation of how to compile R, the manual does a much better job than I could. However, there are some steps that need to be taken to allow R to use the BLAS created above. If you want to try link-time optimization for base R, I will probably address that in a future post.

1. Download and install the latest version of Rtools, including any additional files, if you haven’t already.
   • During the install it is very important to allow Rtools to add its binaries to the PATH, and to delete the MSYS2 entry from the PATH. Otherwise, there will be two versions of GCC related tools in the path, and mayhem may ensue.
   • If Rtools is already installed, make sure to replace the MSYS bin directory in the path with the Rtools one.
2. Re-read the R installation guide, especially if you have compiled R from source for Windows before. There are a lot of changes, such as the need for perl, texi2any, the localsoft directory, ICU, curl, etc.
3. Expand the R source code into R_HOME as described in the manual, including any extra needed files such as the Tcl installed by Rtools, cairodevices, etc.
4. I strongly recommend having a proper install of MIKTEX, Inno, and qpdf as described in the installation manual. Even if you are not interested in creating a full-fledged installer, creating it will help minimize the potential errors found in the checking stage, and thus will make it easier to see if the BLAS is causing a failure.
5. Open R_HOME/src/gnuwin32/MkRules.dist and uncomment and set the following changes, if necessary:
   1. Set USE_ATLAS=YES (Line 30 in R-patched_2016-07-18)
   2. Set ATLAS_PATH (Line 31 in R-patched_2016-07-18) to wherever you have the libopenblas file. For example, if you moved the static library to F:\R\BLAS then set ATLAS_PATH=F:/R/BLAS. Note the forward slashes (/) instead of backslashes (\). This applies to any path in the makefile, such as Localsoft, ICU, cairo, Inno, qpdf, etc.
   3. Set BINPREF64 (Line 59 in R-patched_2016-07-18) to wherever you have Rtools installed. This needs the trailing /! So, if Rtools is installed under F:\, you would set BINPREF64 = F:/Rtools/mingw_64/bin/
   4. Set WIN = 64 (Line 73 in R-patched_2016-07-18)
   5. Uncomment and set the appropriate paths for Localsoft, ICU, curl, Cairo, Inno, qpdf, etc. as necessary
      • Perl is the exception. There /path/to/perl has to terminate in perl.exe. It isn’t just the path, but the full name of the executable.
   6. I would recommend researching the particular machine for which you are compiling, and setting the EOPTS with the appropriate march or at least mtune flag. Using march will make the installation specific to the architecture on which it is built, but the purpose of this exercise is to try and wring as much speed out of the machine on which you are working, is it not? Personally, I use EOPTS = -march=native -pipe. For more information for Rtool‘s version’s GCC settings, see the entries in its documentation.
6. Navigate to and open R_HOME\src\extra\blas\Makefile.win and change line 15 from:-L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lf77blas -latlas to -L../../../$(IMPDIR) -lR -L"$(ATLAS_PATH)" -lopenblas_sandybridgep-r0.2.18 where -lopenblas_sandybridgep-r0.2.18 is the name of the .a file but starting with “-l” and not “lib”.
7. If you are using Rtools34 but compiling R-3.3.x (as I am) you need to make one more change. Navigate to R_HOME/src/gnuwin32/fixed/etc and edit the Makeconf so that TCL_VERSION = 86 (Line 61 in R-patched_2016-07-18). This will not be necessary for R-devel or R-3.4.x.
8. Compile the base R by running make distribution. If there is a significant problem with the BLAS and/or its path, this process may fail. The process may need to be restarted a few times, especially if you get a file not found error. Just run make distribution again.
9. At this point, run make check-all, preferably piping it into a text file (such as make check-all > CheckAll.txt). This can take a long time (20 minutes on a fast machine; over an hour on a slow one). If there is an issue with the BLAS, the check will fail somewhere, although failing the internet check is not a problem. [As of 2015-03-09, there is a bug causing standard internet.dll, but internet2 (and libcurl) work.] I pay careful attention to the times when the check is working on the base, stats, Matrix, and mgcv packages, as most of the time, failures occur in these routines.
10. When completed, check the R_HOME\tests folder and any subdirectory for .Rout.fail files. If you created a log file, then search it for the word ERROR. Many of them will be fine (statements that errors have not been found or the internet failure) but a true error should be found if it exists.
11. If you find a .Rout.fail you may want to try and re-create the error in your newly-compiled R. For example, the only error I get is in example 6, where the call is:
    
    R

    stopifnot( all.equal(fixef(t6.fix.ML.lme), c("(Intercept)"= 24.009565, "I(age - 11)"= 0.64760432), tol= 1e-7) , all.equal(sM6$tTable[,"Std.Error"], c("(Intercept)"= 0.426561, "I(age - 11)"= 0.066832), tol = 5e-5) , all.equal(aM6[,"F-value"], c(3162.47, 93.8969), tol = 5e-5) , all.equal(iM6$varStruct["Female",], c(lower = 0.51230063, est. = 0.65065925, upper = 0.82638482), tol = 5e-5) )

    1 2 3 4 5 6 7 8 9 10 11 12 13 14

    stopifnot(     all.equal(fixef(t6.fix.ML.lme),               c("(Intercept)"= 24.009565, "I(age - 11)"= 0.64760432), tol= 1e-7)     ,     all.equal(sM6$tTable[,"Std.Error"],               c("(Intercept)"= 0.426561, "I(age - 11)"= 0.066832), tol = 5e-5)     ,     all.equal(aM6[,"F-value"], c(3162.47, 93.8969), tol = 5e-5)     ,     all.equal(iM6$varStruct["Female",],       c(lower = 0.51230063, est.  = 0.65065925, upper = 0.82638482), tol = 5e-5) )

    
    and the actual error between iM6$varStruct["Female",] and the constants is 5.720938e-05 > 5e-5. To me this is close enough. Now if the errors were of the order 1e-3 or so, that may be different. It’s your call.

If the check completes and there are no fail files or no significant differences, then congratulations! The Rblas.dll file in R_HOME\bin\x64 is a BLAS tuned to your specific machine.

If anyone does successfully complete this process, I would be very interested to know how much, if at all, the matrix/linear algebra have sped up. Good Luck!