In our overview of general installation, we mentioned a paradigm for when you have to compile from source. As a review, there are typically three steps:
- ./configure sets up environmental variables, checks your system requirements, and well, configures things. Basically, it checks your set up and program lists (in $PATH) to determine how to install things on YOUR SPECIFIC set up. It completes the Makefile to match your file locations, system settings, etc.
- make follows the instructions of the Makefile and converts source code into binary for the computer to read.
- make install installs the program by copying the binaries into the correct places as defined by ./configure and the Makefile. Some Makefiles do extra cleaning and compiling in this step.
If you try this on a cluster, chances are you will run into some issues. The first thing you should try is to add a parameter to ./configure. This is the most common error, so we’re going to point it out before we go through the details. To run ./configure on a system where you have limited access (i.e. a university cluster), you should do the following:
./configure --prefix=<location>
where <location> is where you want the program to be installed. More information on where to put stuff is at the bottom of this post. But first, let’s go over some details on about this configure/make/make install process and how to troubleshoot it.
Break it till you make it: what make is trying to do
Wherever you run make, it will look for a file called Makefile in the current directory. The premise behind a Makefile is similar to a recipe on baking. Where are the ingredients? How do you combine them, and in what order? Can you start somewhere in the middle of the process? How do you clean up?
To approach these issues, Makefiles are organized with some global variables and then a series of directives, like “all”, “install”, “clean”, and more. Makefiles have a certain feel and syntax, but it’s up to the author/distributor of the software what actually goes in them.
The make command, by default, runs the “all” directive in the Makefile. This usually finds the source files (C or C++ files, which usually end in .c) and compiles them into objects (the same file but .o). During compilation, the linker has to find all of the dependencies that the C files require, gather them up, and incorporate all the code.
The last step – make install – copies the binaries into their final locations, might delete intermediates, and does any other step the make step didn’t catch.
make clean is a good idea to run if you want to try compiling all over again. This doesn’t usually destroy the makefile/configuration.
make distclean sets everything back up the way it was when you downloaded it; it may destroy the Makefile itself and the configure caches.
Configure scripts – if there is no Makefile present, you will have to create one by configuring. These are pretty standard, which I really find refreshing. I always use this command structure:
mkdir build
./configure --prefix=$PWD/build
In bash, this means to compile your stuff into your current directory, in the build subdirectory. I like giving it the full path because sometimes configure is finicky about relative paths, the ~ symbol, etc.
If there’s no ‘configure’ file, but there are some files that look like configure.something and a file called autogen.sh, run ./autogen.sh first. This will create your configure executable. If autogen gives you trouble, try autoregen.sh.
Configure will cache some things it finds, such as the locations of libraries or whether you are running on a 64 bit machine. Sometimes, this cache will get in your way and you’ll want to clear it using make distclean. This is especially true if you are missing a library, manage to add it to your library paths, but configure still doesn’t find it. I also keep the original tar.gz or copy of the source somewhere so I can set the whole directory ablaze and start over fresh.
Once you have your configure step finished, usually you can:
make
make install
This usually goes without issues. If you get a permission denied during the make install step, it’s probably trying to write to /usr/bin or /usr/lib. You want it to put the files into a specific spot you have access to, which is what the prefix flag does. If that doesn’t work, you might have to look into the makefile.
Makefile tips
-
If you have a Makefile but no configure script, you’ll have to delve in there and see if the Makefile hard-codes a prefix or path. Look for ‘/usr’ in the Makefile and stealthily change it to whatever path you actually want your stuff installed into. Other key phrases you want to look for are prefix, bin dir, and exec dir.
-
Subdirectories can have their own Makefiles!
-
Use make clean and make distclean (if they exist) between attempts to compile if you changed the Makefile. Otherwise, your half-baked objects can still be there mucking things up.
- Run make -j followed by a number to use multiple threads. If you don’t supply a number, make runs on all cores it can. This can drastically improve the time it takes, but it can cause some Makefiles to puke, and you’ll have to run them without -j. I recommend 4 threads if building on a login node.
If your issue isn’t the location, but some library error, ld error, no such file or directory, etc – you might have to learn a bit about linking.
Cmake config files
Sometimes you will run into Cmake, which is another kind of make paradigm. You need to have cmake installed, or find a Module for it. To use it, you can alter the cmake config files and also give it flags using -D followed by the flag name (no space between). For example
cmake -DFETCH_BOOST=TRUE -
DCMAKE_INSTALL_PREFIX=/some/place/where/I/want/to/build -
DCMAKE_CXX_COMPILER=/N/soft/rhel6/gcc/4.7.2/bin/g++
- flags.make
- build.make
- CMakeLists.txt
Some of those scripts say DO NOT ALTER at the top, but they’re not the boss of you.