This blog post continues our series on software installation without root permissions. For other blog posts on this topic, look for the list under Software installation in this link.
If you are currently or have been installing programs, it is likely that in one of these attempts you would have run into a library error. This blog post walks through what these errors means and how they can be fixed.
Library errors
Why do we run into library errors
Dynamic and Static linking
Compiling dynamic links
Compiling Static links
Fixing library errors
Library errors
So what do library errors look like? Here are a couple of examples to help,
$ pip uninstall drmaa
/N/home/i/u/iugalaxy/Karst/galaxy-17.05/.venv/bin/python2.7: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
What do you think the error is in the above example?
Here is a hint:” error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory”.
Let’s take a look at another example,
$ python3 ~/bin/albacore/usr/lib/python3/dist-packages/albacore/pipeline.py
Traceback (most recent call last):
File “/N/u/cganote/Carbonate/bin/albacore/usr/lib/python3/dist-packages/albacore/pipeline.py”, line 12, in <module>
from albacore.pipeline_core import PipelineCore, get_debug_level as get_core_debug_level
ImportError: libhdf5_cpp.so.11: cannot open shared object file: No such file or directory
What do you think the error is in the above example?
“ ImportError: libhdf5_cpp.so.11: cannot open shared object file: No such file or directory”
In both the examples above, the error suggests the libraries – libpython2.7.so.1.0, libhdf5_cpp.so.11 are not found. This could mean either the library is not installed or that it’s not in your path or environment (click here for more information).
Great, now hopefully that confirms the error you are running into is because of a missing library. If it is, let’s understand what is happening here, in a little more detail and fix the error.
Why do we run into library errors
It really helps when designing software to reuse stuff written by someone else! That way you don’t reinvent the wheel, so to speak. One way to reuse stuff written by someone else is to take their code, copy it all, and paste it right into yours. This is delightful when dealing with 100 lines or so, and a total nightmare at 1,000,000 lines. You can run into issues where you make a mistake copying, or you find out that the other person also has a variable called “i” or a function called “test” and there are conflicts. If the other software comes up with a bug fix or a cool new feature, you would have to update your code to match as well, if you want those nice things.
A way around this is to compile your code into a library that can be shared with other programmers. Shared libraries are files that end in .so and can be included in other C programs. Other languages have this concept too – Python and Perl call them packages or modules. You generally can’t share code between programming languages without special help. Interpreted languages, like Perl and Python, create the machine-ready code at the same time it is run, as opposed to having a separate compiling step. This is done by invoking the perl or python command. You don’t execute a binary by putting “C” in front of it first – you just run it – but for interpreted languages, you have to give it the command in front, or the first line of the program has to have a path to the interpreter like #!/usr/bin/python. The interpreter knows how to speak the language and turn it into runnable instructions. See more detail here. This is great for portability since the Python on my machine should know what to do with a print command you wrote on your machine. The final commands the machine gets may differ between our machines, and that’s ok! Java really took it a step further by setting up its own virtual machine to run its code in, as well as manage memory space and isolate itself. This added stuff costs you performance, though, and a lot of Java programs are heavy resource eaters compared to other languages.
Linking and the art of figure-it-out-later
Great, so you can include code from other programs written in your language. In C, there’s an additional consideration. Once you pulled the code from this other source into your repertoire, how do you combine these codes,
- do you integrate it into your binary?
- Or do you keep it aside as a reference, so that when your binary is run later, you can go and find it again?
This decision is made when you compile your code – you can choose to keep it flexible with Dynamic linking, or you can choose to package it all together with a Static build of your code. Generally, dynamic linking is preferred. This is because when you give your binary to someone else, they can use their spiffy, optimized-to-all-heck math libraries and make your code run fast on their machine. As another example, if you are including some security-sensitive code in your program, like SSL libraries, you don’t want to wrap up the code in your binary that might have critical bug fixes later. If someone hacks the SSL library version that’s in your code, you have to recompile it to include a new version. If you linked it dynamically, you just recompile SSL and all the code that uses it will use the updated version! Additionally, the more dependencies you statically link, the bigger your file gets. This might not be an issue for a few things, but if every program you install has its own version of the whole C library inside, that becomes a lot of redundant code you are housing. You will have to weigh these considerations against the convenience of putting it all in one place and distributing it as a complete package. If you want more gory details, read through this nice article.
How to compile dynamic links
Usually, you can add the -fPIC flag to your compiler. You can alter your CFLAGS environment variable to give options to the C compiler, but it’s up to the Makefile if it even bothers to look at the environment. The same thing goes for your library paths. Note that CPP does not mean C plus plus. It means C pre-processor. This can be useful to change in rare cases. If you are working with C++, use CXXFLAGS instead of CFLAGS.
A few other things you can try:
- –enable-shared during configure
- CRAYPE_LINK_TYPE=dynamic if you are working on a Cray
- option (BUILD_SHARED_LIB “Build a shared library” ON) in CMakeLists.txt
Here’s a blurb you might see at the end of your make install:
- add LIBDIR to the `LD_LIBRARY_PATH’ environment variable during execution
- add LIBDIR to the `LD_RUN_PATH’ environment variable during linking
- use the `-Wl,-rpath -Wl,LIBDIR’ linker flag
Do those things =)
Compiling statically
You can take out the PIC flag, so no -fPIC in your Makefile or environment. Also, try –enable-static in your configure line.
Messing with the linker
If you want to see which libraries are dynamically linked to a binary, type ldd followed by the full path to that binary. Just for fun, let’s see what libraries the linker itself needs!
ldd /usr/bin/ld
linux-vdso.so.1 => (0x00007fff6510c000)
libbfd-2.27-28.base.el7_5.1.so => /usr/lib64/libbfd-2.27-28.base.el7_5.1.so (0x00002b8ba591e000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x00002b8ba5c65000)
libc.so.6 => /usr/lib64/libc.so.6 (0x00002b8ba5e69000)
/lib64/ld-linux-x86-64.so.2 (0x00002b8ba56fa000)
If you see ⇒ not found on one of the lines, there’s a good chance you’ll have trouble running that program; the library isn’t being found in your current environment, like the examples mentioned in the above section.
When searching for a library you need, you might ask the linker directly. For example, let’s say I’m looking for the BLAS math library. The file I need will start with lib, followed by the library name, then .so and possibly some version numbers. For example, on my system, the blas library is found in:
/lib64/libblas.so.3
If you want to link to blas, you need to use the -l flag, followed by the library name. For example:
ld -lblas –verbose
ld only searches the directories it is configured to search. It lists those at the end if you use –verbose. Adding a directory to ld’s default path is a root action (ldconfig), so we can’t do that. If there is no library in ld’s default search paths, you can try using the compiler itself. First, get yourself some C code to test linking with. Here’s a little C program. Let’s name it testImport.c.
#include <stdio.h>
int main(){
printf (“Hello Solar System\n”);
}
Once you save that to a file, test compiling it while linking the blas library:
gcc -lblas -o testImport testImport.c
If that happens with no error, that means that your library was able to be linked! You can then see where the library was located using ldd:
ldd testImport
linux-vdso.so.1 => (0x00007ffd26acc000)
libblas.so.3 => /lib64/libblas.so.3 (0x00002b51542dc000)
libc.so.6 => /lib64/libc.so.6 (0x00002b5154535000)
You can use gcc like this to experiment with different compiler flags and environment variables. The -L flag can let you specify which directory your library is in, if ld can’t find it and you don’t have luck putting it in environment variables like LIBRARY_PATH.
gcc -L/N/u/cganote/Carbonate/bin/rmats -lblas -o testImport testImport.c
Note, however, when I invoke ldd on this:
ldd testImport
linux-vdso.so.1 => (0x00007ffd4d174000)
libblas.so.3 => /lib64/libblas.so.3 (0x00002b0c21066000)
libc.so.6 => /lib64/libc.so.6 (0x00002b0c212bf000)
The location of blas hasn’t changed! That’s because the -L flag only applies to gcc. Once the binary is created, if the library is dynamically linked, it has to find it all over again, and it forgot all about whatever you put in -L while it was being built.
I can press the issue on gcc using the Wl,-R flag. Here’s what that looks like:
gcc -Wl,-R/N/u/cganote/Carbonate/bin/rmats -lblas -o testImport testImport.c
ldd testImport
linux-vdso.so.1 => (0x00007ffde1d76000)
libblas.so.3 => /N/u/cganote/Carbonate/bin/rmats/libblas.so.3 (0x00002af206980000)
It remembered! However, this can cause issues if you move the directory with libblas.so in it. Let’s say I change the name:
mv rmats rmat5
ldd testImport
linux-vdso.so.1 => (0x00007ffd27398000)
libblas.so.3 => /lib64/libblas.so.3 (0x00002aab0ad73000)
Since the -R path with blas was no longer there, the linker had to go find blas again. If you don’t have another copy of that library somewhere in your LD_LIBRARY_PATH or in the usual places ld looks, your program will break. This also makes it less convenient to compile in one place and then move the libraries around willy-nilly.
If you are still running into a library error, email us at help@ncgas.org.