Building R-devel on RedHat Linux 6

Warning: I’m 85% done with this, formatting is not right. I DO NOT want to type in the prompt in front of every command because then one cannot copy/paste directly.  However, copying some output chunks picks up the dollar signs and I’m inconsistent.

Brief Summary: R-devel will not build without access to newer zlib, bzip2.  This is a problem because CRAN requires users to test packages against R-devel, not against existing R or R-patched. This note has step-by-step information about what was necessary to compile & install R-devel in my user account on a cluster compute environment running RedHat 6 Linux.To upload an R package, one must agree to compile the package against R-devel, the cutting edge version of R.

The 2016 current version of R-devel has removed the versions of several compression libraries that used to be included. Instead of providing those libraries, R-devel supposes they are installed in the operating system. On my up-to-date Ubuntu laptop, this was not a concern because I have up-to-date versions of zlib, xz, pcre, and curl.On the compute cluster, which is still running RedHat 6, it is a more serious problem because the libraries zlib, bzip, pcre, curl, and xz are out of date. We find that out because when we try to build R-devel, it fails and tells us what is out of date.

As a result, one cannot configure and compile R-devel. One must get updated libraries. If the system administrators would replace all of those libraries, we could go ahead.However, in a Unix system, it is possible to compile and install support libraries in a user’s account, without system-wide intervention. With the exception of bzip2, where theinstallation is a non-standard setup, the installs of zlib, xz, curl, and pcre are standard and easy. Building R-devel on a Linux system with slightly older packges.

Here is the process I went through on RHEL6 to make this go. Special thanks to Wes Mason at KU ITTC who provided the critical ingredient.

1. Our cluster defaults to an ancient version of gcc.   I can tell my environment to use the newer gcc compiler

$ module avail$ module load gcc/4.9.22

2. Try to build R-devel without making any special preparations.

mkdir src cd src wget --no-check-certificate https://stat.ethz.ch/R/daily/R-devel_2016-02-11.tar.gz tar xzvf R-devel_2016-02-11.tar.gz cd R-devel ./configure --help

mkdir builddir cd builddir ../configure --prefix=$HOME/packages/R-devel '--with-cairo' \ '--with-jpeglib' '--with-readline' '--with-tcltk' \ '--with-blas' '--with-lapack' '--enable-R-profiling' \ '--enable-R-shlib' \ '--enable-memory-profiling'

## fails ignominously: checking if zlib version >= 1.2.5... no checking whether zlib support suffices... configure: error: zlib library and headers are required

3. Install zlibDownload, un-tar, configure, compile

cd ~/src wget http://zlib.net/zlib-1.2.8.tar.gz tar xzvf zlib-1.2.8.tar.gz cd zlib-1.2.8 ./configure --prefix=$HOME/packages

I won’t show all the output for all of these things, but this is brief and representative

 Checking for gcc... Checking for shared library support... Building shared library libz.so.1.2.8 with gcc. Checking for off64_t... Yes. Checking for fseeko... Yes. Checking for strerror... Yes. Checking for unistd.h... Yes. Checking for stdarg.h... Yes. Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf(). Checking for vsnprintf() in stdio.h... Yes. Checking for return value of vsnprintf()... Yes. Checking for attribute(visibility) support... Yes.

This is the common GNU-style software. configure. make. make install. Run those:

makemake install

$ make install cp libz.a /home/pauljohn/packages/lib chmod 644 /home/pauljohn/packages/lib/libz.a cp libz.so.1.2.8 /home/pauljohn/packages/lib chmod 755 /home/pauljohn/packages/lib/libz.so.1.2.8 cp zlib.3 /home/pauljohn/packages/share/man/man3 chmod 644 /home/pauljohn/packages/share/man/man3/zlib.3 cp zlib.pc /home/pauljohn/packages/lib/pkgconfig chmod 644 /home/pauljohn/packages/lib/pkgconfig/zlib.pc cp zlib.h zconf.h /home/pauljohn/packages/include chmod 644 /home/pauljohn/packages/include/zlib.h /home/pauljohn/packages/include/zconf.h

4. Adjust the environment so R-devel builds will find packages installed there.

 export PATH=$HOME/packages/bin:$PATH export LD_LIBRARY_PATH=$HOME/packages/lib:$LD_LIBRARY_PATH  export CFLAGS="-I$HOME/packages/include"  export LDFLAGS="-L$HOME/packages/lib" 

The first two are vital during the “make” phase in R-devel, the latter 2 are vital in the “configure” phase in R-devel.5. Try to build R-devel again, using new zlibI remove and remake the build directory, so that any accumulated errors are eliminated

cd ~/srccd R-devel/rm -rf builddirmkdir builddircd builddir/

../configure --prefix=$HOME/packages/R-devel --with-cairo \ --with-jpeglib --with-readline --with-tcltk \ --with-blas --enable-BLAS-shlib --with-lapack --enable-R-profiling \ '--enable-R-shlib' \ '--enable-memory-profiling'

That succeeds, finds zlib, but configure ends with this error:

checking bzlib.h presence... yes checking for bzlib.h... yes checking if bzip2 version >= 1.0.6... no checking whether bzip2 support suffices... configure: error: bzip2 library and headers are required

6. Get new bzlib support. This one is not built with GNU auto tools,so it is a little more interesting/idiosyncratic. Would not havesolved it without help from this site:http://www.linuxfromscratch.org/lfs/view/development/chapter06/bzip2.html

## So we go get bzlib, which is part of bzip2, just like we did on zlib

cd ~/srcwget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gztar xzvf bzip2-1.0.6.tar.gzcd bzip2-1.0.6

Inspect the README. Temptation is to be careless and just run make, but that’s not quite enough because we need the shared library. Then make after

make -f Makefile-libbz2_so make clean make make -n install PREFIX=$HOME/packages make install PREFIX=$HOME/packages

7. Try to build R-devel again

cd ~/src/R-devel rm -rf builddir/ mkdir builddir cd builddir ../configure --prefix=$HOME/packages/R-devel '--with-cairo' \ '--with-jpeglib' '--with-readline' '--with-tcltk' \ '--with-blas' '--with-lapack' '--enable-R-profiling' \ '--enable-R-shlib' \ '--enable-memory-profiling'

configure fails with

checking whether bzip2 support suffices... no checking for lzma_version_number in -llzma... no configure: error: "liblzma library and headers are required"

8. Go get liblzma. I tried that, couldn’t compile that, but Wes Mason warned me not to try to install the separate liblzma, but rather get the package known as xz.

cd ~/srcwget http://tukaani.org/xz/xz-5.2.2.tar.gztar xzvf xz-5.2.2.tar.gzcd xz-5.2.2./configure --prefix=$HOME/packagesmake -j3make install

8.  Try R-devel again, same steps as before, make builddir, then

../configure --prefix=$HOME/packages/R-devel '--with-cairo' \ '--with-jpeglib' '--with-readline' '--with-tcltk' \ '--with-blas' '--with-lapack' '--enable-R-profiling' \ '--enable-R-shlib' \ '--enable-memory-profiling'

That gets quite a bit further and fails:

checking for pcre/pcre.h... no checking if PCRE version >= 8.10, < 10.0 and has UTF-8 support... no checking whether PCRE support suffices... configure: error: pcre >= 8.10 library and headers are required

9. Get pcre. This is getting old now

cd ~/src wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.38.tar.gz tar xzvf pcre-8.38.tar.gz ./configure --prefix=$HOME/packages make -j3 make install

9B. Back to R-develR-devel configure fails same way:

checking for pcre/pcre.h... nochecking if PCRE version >= 8.10, < 10.0 and has UTF-8 support...no checking whether PCRE support suffices... configure: error: pcre >= 8.10 library and headers are required

9C So I suspect the UTF-8 support is the issue.Back to PCRE, reconfigure. Usually, best to erase whole sourcedirectory and get a clean run at it. Because I did not do safe thingand use a builddir in there, I have to do that. I run this configure command,

./configure --enable-utf8 --prefix=$HOME/packagesmake make install

10. Try R-devel again.Fails asking for libcurl

 checking libcurl version ... 7.19.7 checking curl/curl.h usability... yes checking curl/curl.h presence... yes checking for curl/curl.h... yes checking if libcurl is version 7 and >= 7.28.0... no configure: error: libcurl >= 7.28.0 library and headers are required with support for https

11. Install libcurl## Note need ignore certificate problem on this one

 cd ~/src wget --no-check-certificate https://curl.haxx.se/download/curl-7.47.1.tar.gz tar xzvf curl-7.47.1.tar.gz cd curl-7.47.1 ./configure --prefix=$HOME/packages make -j3 make install

12. Try R-devel again

cd ~/src cd R-devel rm -rf builddir mkdir builddir cd builddir ../configure --prefix=$HOME/packages/R-devel '--with-cairo' \ '--with-jpeglib' '--with-readline' '--with-tcltk' \ '--with-blas' '--with-lapack' '--enable-R-profiling' \ '--enable-R-shlib' \ '--enable-memory-profiling'

HOORAY, it finished!

config.status: creating tests/Embedding/Makefile config.status: creating tests/Examples/Makefile config.status: creating tools/Makefile config.status: creating src/include/config.h config.status: executing libtool commands config.status: executing stamp-h commands

R is now configured for x86_64-pc-linux-gnuSource directory: .. Installation directory: /home/pauljohn/packages/R-develC compiler: gcc -std=gnu99 -I/home/pauljohn/packages/include Fortran 77 compiler: gfortran -g -O2C++ compiler: g++ -g -O2 C++11 compiler: g++ -std=c++11 -g -O2 Fortran 90/95 compiler: gfortran -g -O2 Obj-C compiler: gcc -g -O2 -fobjc-exceptionsInterfaces supported: X11, tcltk External libraries: readline, BLAS(generic), LAPACK(generic), curl Additional capabilities: PNG, JPEG, NLS, cairo, ICU Options enabled: shared R library, R profiling, memory profilingCapabilities skipped: TIFF Options not enabled: shared BLASRecommended packages: yes

configure: WARNING: you cannot build info or HTML versions of the R manualsconfigure: WARNING: neither inconsolata.sty nor zi4.sty found: PDF vignettes and package manuals will not be rendered optimally

That last warning, well, I’m ignoring it. I don’t need to build their documents, I need to see if my package builds without errors. I don’t care much that shared BLAS is not enabled, but I ususally would want that if I were making a production system.However, running

make

ends in failure:

gcc -std=gnu99 -shared -fopenmp -L/home/pauljohn/packages/lib -o libR.so CommandLineArgs.o Rdynload.o Renviron.o RNG.o agrep.o apply.o arithmetic.o array.o attrib.o bind.o builtin.o character.o coerce.o colors.o complex.o connections.o context.o cum.o dcf.o datetime.o debug.o deparse.o devices.o dotcode.o dounzip.o dstruct.o duplicate.o edit.o engine.o envir.o errors.o eval.o format.o gevents.o gram.o gram-ex.o graphics.o grep.o identical.o inlined.o inspect.o internet.o iosupport.o lapack.o list.o localecharset.o logic.o main.o mapply.o match.o memory.o names.o objects.o options.o paste.o platform.o plot.o plot3d.o plotmath.o print.o printarray.o printvector.o printutils.o qsort.o radixsort.o random.o raw.o registration.o relop.o rlocale.o saveload.o scan.o seq.o serialize.o sort.o source.o split.o sprintf.o startup.o subassign.o subscript.o subset.o summary.o sysutils.o times.o unique.o util.o version.o g_alab_her.o g_cntrlify.o g_fontdb.o g_her_glyph.o xxxpr.o `ls ../unix/*.o ../appl/*.o ../nmath/*.o` ../extra/tre/libtre.a -lblas -lgfortran -lm -lquadmath -lreadline -lpcre -llzma -lbz2 -lz -lrt -ldl -lm -licuuc -licui18n /usr/bin/ld: /home/pauljohn/packages/lib/libbz2.a(bzlib.o): relocation R_X86_64_32S against `BZ2_crc32Table' can not be used when making a shared object; recompile with -fPIC /home/pauljohn/packages/lib/libbz2.a: could not read symbols: Bad value collect2: error: ld returned 1 exit status make[3]: *** [libR.so] Error 1 make[3]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/main' make[2]: *** [R] Error 2 make[2]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/main' make[1]: *** [R] Error 1 make[1]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src' make: *** [R] Error 1

13. That’s certainly pointing the finger back at bzip2, which is the only non-standard library in the whole batch. It doesn’t use GNU autoconf, has vague instructions. I went into the bzip2 directory and inserted -fPIC as a CFLAG in the Makefile. Then I ran make and make install PREFIX=$HOME/packages again, as above14. R-devel, againrm the builddirmake a new builddir, go in there, run the configure statement, looksOK

make

succeeds. Be aware, it is VITAL the PATH and LD_LIBRARY_PATH be set in the environment as stated above.Here’s the evidence it did eventually compile

make[2]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/library/Recommended'make[1]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/library/Recommended'make[1]: Entering directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/library'building/updating vignettes for package 'grid' ...building/updating vignettes for package 'parallel' ...building/updating vignettes for package 'utils' ...make[1]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir/src/library'make[1]: Entering directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir'configuring Java ...Java interpreter : /usr/bin/javaJava version : 1.7.0_09-icedteaJava home path : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.9.x86_64/jreJava compiler : /usr/bin/javacJava headers gen.: /usr/bin/javahJava archive tool: /usr/bin/jar

trying to compile and link a JNI program detected JNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/linuxdetected JNI linker flags : -L$(JAVA_HOME)/lib/amd64/server -ljvmmake[2]: Entering directory `/library/tmp/Rjavareconf.wg86X6'gcc -std=gnu99 -I/home/pauljohn/src/R-devel/builddir/include -I/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.9.x86_64/jre/../include -I/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.9.x86_64/jre/../include/linux -I/usr/local/include -fpic -I/home/pauljohn/packages/include -c conftest.c -o conftest.ogcc -std=gnu99 -shared -L/home/pauljohn/src/R-devel/builddir/lib -L/home/pauljohn/packages/lib -o conftest.so conftest.o -L/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.9.x86_64/jre/lib/amd64/server -ljvm -L/home/pauljohn/src/R-devel/builddir/lib -lRmake[2]: Leaving directory `/library/tmp/Rjavareconf.wg86X6'

JAVA_HOME : /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.9.x86_64/jreJava library path: $(JAVA_HOME)/lib/amd64/serverJNI cpp flags : -I$(JAVA_HOME)/../include -I$(JAVA_HOME)/../include/linuxJNI linker flags : -L$(JAVA_HOME)/lib/amd64/server -ljvmUpdating Java configuration in /home/pauljohn/src/R-devel/builddirDone.

make[1]: Leaving directory `/panfs/pfs.acf.ku.edu/home/pauljohn/src/R-devel/builddir'