[Previous] [Up] [Next]

A Tour of NTL: Obtaining and Installing NTL for UNIX

This procedure should work on most Unix or Unix-like platorms (including Mac OS, and Windows with MinGW or Cygwin tools).

To obtain the source code and documentation for NTL, download ntl-xxx.tar.gz, placing it a directory, and then, working in this directory, do the following. Here, "xxx" denotes the current version number.

   % gunzip ntl-xxx.tar.gz
   % tar xf ntl-xxx.tar
   % cd ntl-xxx/src
   % ./configure 
   % make
   % make check
   % sudo make install
This will build, test, and install NTL in /usr/local. For this to work, GMP must already be installed (most Unix distributions already come with GMP installed, but see this page for more details). If you really do not want to use GMP, you can pass the option NTL_GMP_LIP=off to configure.

After installation, you will find the NTL header files in /usr/local/include/NTL and the compiled binary in /usr/local/lib/libntl.a (this is a static library -- if you want a shared library, see below). Documentation is in /usr/local/share/doc, with the main entry-point at /usr/local/share/doc/tour.html.

If you want very high-performance for polynomial arithmetic over GF(2), you may want to consider using the gf2x library. To do this, gf2x must already be installed. In addition, you should invoke configure with the option NTL_GF2X_LIB=on. This page provides more details.

If you want to install NTL somewhere besides /usr/local, pass the option PREFIX=/path/to/install/ntl to configure. If you do this, you may not need to run make install using sudo. If GMP is installed somewhere besides /usr/local, pass the optopn GMP_PREFIX=/path/to/gmp to configure. You can also pass GF2X_PREFIX=/path/to/gf2x to configure, if gf2x is installed somewhere besides /usr/local.

As a shorthand, you pass the option DEF_PREFIX=/path/to/all/software, which will override the default for PREFIX, GMP_PREFIX, and GF2X_PREFIX.

Now suppose you want to compile a program that uses NTL. Suppose you are working in some arbitrary directory and foo.cpp is your program. Assume that you have installed NTL in /usr/local as above. The following should work:

   % g++ -g -O2 foo.cpp -o foo -lntl -lgmp -lm
If you have installed NTL and/or GMP is a non-standard location, say /path/to/sw, then:
   % g++ -g -O2 -I/path/to/sw/include foo.cpp -o foo  -L/path/to/sw/lib -lntl -lgmp -lm
If you build NTL with gf2x, just add the option -lgf2x to the above, right after -lgmp.

If you are working in the NTL src directory itself, you can just run:

   % make foo
to compile a program foo.cpp, as above.

More Details

What follows is a more detailed description of the installation process.

Step 1. Extract the source files by executing:

   % gunzip ntl-xxx.tar.gz
   % tar xvf ntl-xxx.tar
On most systems, the following shortcut works:
   % tar xzvf ntl-xxx.tar.gz

Note that this will unpack everything into a sub-directory ntl-xxx, creating this directory if necessary. Next:

   % cd ntl-xxx
   % ls
You should see a file "README", and directories "include", "doc", and "src". The directory "doc" contains all the documentation. The file "doc/tour.html" contains a copy of the on-line documentation. The directory "include" contains all the header files within a subdirectory "include/NTL". The directory "src" contains everything else. Go there now:
   % cd src

Step 2. Run the configuration script.

Execute the command

   % ./configure [ variable=value ]...
This configure script generates the file "makefile" and the file "../include/NTL/config.h", based upon the values assigned to the variables on the command line.

Here are the most important variables, and their default values.

CXX=g++              # The C++ compiler

CXXFLAGS=-g -O2      # C++ complilation flags

NATIVE=on            # Compiles code targeted to the current hardware (see below)
TUNE=auto            # Performance-tuning switch (see below)

DEF_PREFIX=/usr/local# Default software directory

PREFIX=$(DEF_PREFIX) # Directory in which to install NTL library components
SHARED=off           # Generate a shared library (as well as static -- see below)

NTL_THREADS=off      # compile in thread-safe mode (see below)
NTL_THREAD_BOOST=off # compile with thread boosting enabled (see below)
NTL_EXCEPTIONS=off   # compile with exceptions enabled (see below)

NTL_GMP_LIP=on       # Switch to enable the use of GMP as primary 
                     #   long integer package

GMP_PREFIX=$(DEF_PREFIX) # Directory in which GMP components are installed

NTL_GF2X_LIB=off     # Switch to enable the use of the gf2x package
                     #   for faster arithmetic GF(2)[X]

GF2X_PREFIX=$(DEF_PREFIX) # Directory in which gf2x components are installed


Some magic. The special makefile variable CXXAUTOFLAGS is automagically set by the configuration script. These are C++ compiler flags that are selected depending on the choice of other configuration options, and is geared towards gcc. The configuration script always tests that these flags actually work, and prints out the flags that it chooses. If you explicitly set CXXAUTOFLAGS when invoking the configuration script, then it will not change that value.

To set CXXAUTOFLAGS, the configure script actually needs to run make. If you wish to use a non-standard make program for this purpose, say gmake, set MAKE_PROG=gmake.

Going native. If NATIVE=on (the default), then using the CXXAUTOFLAGS mechanism, the flag -march=native gets passed to the compiler (assuming the compiler accepts that flag). Some users may choose to override this mechanism by setting NATIVE=off, since -march=native can cause trouble in some environments. The default, however, is NATIVE=on, because otherwise most users will end up not benefiting from important optimizations. If you do opt to set NATIVE=off, please consider adding more specific flags to CXXFLAGS, such as

These are by far the most important instructions that NTL currently exploits, so they should be enabled if at all possible. Alternatively, you could add -march=XXX to CXXFLAGS to enable instructions for a specific CPU type (e.g., XXX=core-avx2 works for Haswell CPUs).

The question of "fat builds" for NTL has occasionally come up. While that is not possible at the moment, it should still be possible to build different versions of NTL targeted for different micro-architectures. NTL's include files generally do not depend on the micro-architecture, so it should be possible to compile client code against a single set of header files, and then dynamically link against the approprate version of the library.

Tuning options. If TUNING=auto (the default), then make will run a performance-tuning "wizard" to set a number of fine-grained performance options. This is highly recommended, as it will choose options that are best suited to your particular hardware and compiler; however, it can be a bit time consuming. Alternatively, you can set TUNING to one of the following values.

More choices may be added in the future.

Esoterica. There are a number of more esoteric configuration variables that can be set. See config.txt for a complete description.

Skipping this step. Note that the distribution comes with pre-installed makefile and config.h files, so it is possible to skip the configure step entirely. However, this is usually a bad idea.

Step 3. Execute make.

Just type:

   % make

The build process after this point is fully automatic. But here is a description of what happens.

  1. The makefile builds the file "../include/NTL/mach_desc.h", which defines some machine characteristics such as word size and floating-point precision. This is done by compiling and running a C++ program called MakeDesc that figures out these characteristics on its own, and prints some diagnostics to the terminal.

  2. Several scripts are run to obtain more information about your system (e.g., to find a timing function, a "getpid" function, and to detect if things like Intel AVX intrinsics work).

  3. The file "../include/NTL/gmp_aux.h" is generated for use with GMP. If not using GMP, this file is still created, but it is empty.

  4. If TUNE=auto (the default), the configuration wizard script is run. This script works in a sub-directory, compiling several programs, and performing a number of timing experiments, in order to determine the optimal setting for a number of flags in the file ../include/NTL/config.h. When the script finishes (it may take several minutes), you will be told what the wizard thinks are the best settings, and your config.h file will be automatically updated. Note that any flags you set in Step 2 will be in effect while the wizard runs, and will be retained in the updated config.h file, with the exception of the flags
    which are set by the wizard.

  5. The makefile will compile all the source files, and then creates the library "ntl.a" in the current directory.

Note that for finer control you can optionally break up this process into the five component steps:

   % make setup1
   % make setup2
   % make setup3
   % make setup4
   % make ntl.a

After NTL is built.

Executing make check runs a series of timing and test programs. It is a good idea to run this to see if everything really went well. It may take a few minutes.

Executing make install copies a number of files to a directory <prefix> that you specify by passing PREFIX=<prefix> as an argument to configure at configuration time. The default is /usr/local, so either you need root permissions, or you choose a <prefix> for which you have write permission. The files ../include/NTL/* are copied into <prefix>/include/NTL. The file ntl.a is copied to <prefix>/lib/libntl.a. The files ../doc/* are copied into <prefix>/share/doc/NTL.

You can also "fine tune" the installation procedure further. See the configure documentation for details.

To allow for the "staging area" trick that some package managers use, the makefile uses a vatriable DESTDIR that is prepended to PREFIX at installation time. So executing

   % make install DESTDIR=/path/to/staging/area
will install the components into the staging area. Note that DESTDIR is set to empty by default, and is not set by the configure script.

Executing make uninstall undoes make install.

Executing make clobber essentially undoes make. Make sure you do this if you re-build NTL for a different architecture!

Executing make clean will remove object files, but not ntl.a. To rebuild after executing make clean, execute make ntl.a.

Building a Shared Library

By default, the above installation procedure builds a static library only. Static libraries are nice because the procedures for building and using them are nearly identical across various flavors of Unix. However, static libraries have their drawbacks, and sometimes it is desirable to build a shared library. This can be done (in theory) by simply passing SHARED=on to NTL's configure.

If you set SHARED=on, then behind the scenes, the procedure used by the makefile changes a bit. In particular, the magical program libtool is used to deal with all idiosyncracies of shared libraries. By default, the makefile configures and uses its own libtool script. You can override this behavior by setting configuration variable LIBTOOL to point to another version of libtool. Note that if SHARED=on, then in addition to using the libtool program, the makefile relies on features specific to GNU make.

On Cygwin, you may need to set LIBTOOL_LINK_FLAGS=-no-undefined for very obscure reasons.

Note that if you want to build NTL as a shared library, then (if you use them) GMP and gf2x must also be built and installed as shared libraries. Also note that to use a shared library version of NTL, you may have to do something special, like set a special shell variable: the output generated by the libtool program during make install should give specific instructions. In addition, if NTL is built as a shared library, then you typically do not have to include -lgmp (if using GMP), or -lgf2x (if using gf2x), or corresponding -L flags, or -lm on the command line when compiling programs that use NTL.

Thread safety, thread boosting, and exception safety

By default, NTL is currently not thread safe. To make it so, you must configure with NTL_THREADS=on. However, this requires a C++11 compiler. Currently, the configure script will try to add -std=c++11 and -pthread to CXXAUTOFLAGS, as this is what gcc requires.

You might also consider configuring with NTL_THREAD_BOOST=on to make NTL internally use a thread pool to boost performance. This also makes the same thread-boosting facility available to NTL client code. Note that NTL_THREAD_BOOST=on implies NTL_THREADS=on.

Note that GMP is thread safe, but (currently) gf2x is not. So you cannot use gf2x in a thread-safe build of NTL.

By default, NTL treats errors in a very old-fashioned way: by printing an error message and aborting your program. However, you can override this behavior by configuring with NTL_EXCEPTIONS=on. This will make NTL throw an exception instead of aborting. This feature also requires a C++11 compiler (and CXXAUTOFLAGS is adjusted appropriately).

[See here] for more details about thread safety and exceptions.

[See here] for more information about thread pools and thread boosting.

32-bit and 64-bit ABIs

An ABI (Application Binary Interface) defines the sizes of various C data types. Typically, with a 32-bit ABI, int's and long's are 32 bits, while on a 64-bit ABI, int's are 32 bits and long's are 64 bits. Some platforms support both 64-bit and 32-bit ABI's; typically in such settings, the 64-bit ABI will yield much better performance, while the 32-bit ABI is available for backward compatibility. On modern 64-bit platforms, the 64-bit ABI is typically the default; otherwise, you may have to explicitly request it (e.g., with gcc, pass the -m64 flag to the compiler).

If you are using NTL with either the GMP or gf2x libraries, then these must be built with the same ABI as NTL. The installation script for GMP will typically select the 64-bit ABI automatically if it is available. The installation script for gf2x may need some hints.

When compiling programs that use NTL, you must also ensure that the program is compiled with the same ABI as NTL.

[Previous] [Up] [Next]