Copyright (C) 2009-2012 Peter Breitenlohner You may freely use, modify and/or distribute this file. Design of the TeX Live (TL) build system ======================================== The TL build system has been completely redesigned, using Autoconf, Automake, and Libtool. The main components are TeX specific or utility program modules (directories texk/*/ and utils/*/), and TeX specific or generic library modules (directories texk/kpathsea/, texk/ptexenc/, and libs/*/) providing libraries required by the program modules. The primary design goal was modularity. Each module specifies its own requirements and properties, such as required libraries, whether an installed (system) version of a library can be used, configure options to be seen at the top-level, and more. Consequently an explicit list of all available modules is kept only in one central place. A second, related goal was to configure and build each library before configuring any (program or other library) module depending on that library. This allows to check for properties and features of a library built as part of the TL tree in much the same way as for a system version of that library. Most generic libraries and several programs that are maintained independently use the distributed source tree, with any patches to that source tree also kept separately as precise documentation of our changes. All this should simplify upgrading of modules maintained independently and/or integrating new modules into the TL build system. Layout of the TL build system ============================= 1. Layout of directories ======================== 1.1. Overall layout ------------------- texk/kpathsea/ the two TeX specific libraries, Kpathsea texk/ptexenc/ and Ptexenc libs/Lib/ generic library Lib (zlib, libpng, t1lib, etc.) texk/Prog/ TeX specific program package Prog (web2c, afm2pl, etc.) utils/Util/ utility program package Util (chktex, ps2eps, etc.) auxdir/auxsub/ dummy module used to configure the real modules */*/ac/*.ac various configure.ac fragments (configure options) build-aux/ auxiliary build scripts (config.guess etc.) m4/kpse-pkgs.m4 Autoconf macros defining lists of library and program modules and handling loops over these lists m4/*.m4 other Autoconf macros texk/am/*.am Makefile.am fragments texk/web2c/am/*.am for the Kpathsea library texk/web2c/*/am/*.am and the Web2C programs 1.1.1. The TeX specific libraries texk/kpathsea/ac/withenable.ac defines that kpathsea requires no other libraries texk/kpathsea/ac/kpathsea.ac defines basic tests for a system version of kpathsea texk/kpathsea/ac/mktex.ac configure options also seen at the top-level m4/kpse-kpathsea-flags.m4 defines Autoconf macros for configure options as well as KPSE_KPATHSEA_FLAGS defining Make variables and to be used in configure.ac files for modules depending on kpathsea. texk/ptexenc/ac/withenable.ac defines that ptexenc requires kpathsea texk/ptexenc/ac/ptexenc.ac defines basic tests for a system version of ptexenc m4/kpse-ptexenc-flags.m4 defines Autoconf macros for configure options as well as KPSE_PTEXENC_FLAGS defining Make variables and to be used in configure.ac files for modules depending on ptexenc. 1.1.2 A generic library `Lib' libs/Lib/ac/withenable.ac defines libraries required by Lib (if any) libs/Lib/ac/Lib.ac defines basic tests for a system version of Lib m4/kpse-Lib-flags.m4 defines Autoconf macros for configure options as well as KPSE_LIB_FLAGS defining Make variables and to be used in configure.ac files for modules depending on this library. 1.1.3. A TeX specific program `Prog' texk/Prog/ac/withenable.ac defines libraries required by Prog, provides the configure option --disable-Prog texk/Prog/ac/Prog.ac optionally defines Prog specific configure option that should be seen at the top-level 1.1.4. A utility program `Util' utils/Util/ac/withenable.ac defines libraries required by Util, provides the configure option --disable-Util 1.2. The individual modules --------------------------- All generic libraries and many programs are maintained independently and use the distributed source tree, with any patches to that source tree kept as documentation of our changes. There are several types of proxy or wrapper build systems for these modules. 1.2.1. A proxy build system, e.g., for libs/zlib/ libs/zlib/zlib-1.2.7/ (almost) unmodified source tree libs/zlib/zlib-1.2.7-PATCHES/ modifications applied to that source tree libs/zlib/Makefile.am proxy Makefile.am libs/zlib/configure.ac and configure.ac In these cases the build system of the distributed source tree is ignored. The header files for many libraries are `installed' (as symlinks) under libs/Lib/include/ in the build tree exactly as they are for a system version under, e.g., /usr/include/. 1.2.2. A wrapper build system, e.g., for libs/freetype2/ libs/freetype2/freetype-2.4.9/ (almost) unmodified source tree libs/freetype2/freetype-2.4.9-PATCHES/ modifications applied to that source tree libs/freetype2/Makefile.am wrapper Makefile.am libs/freetype2/configure.ac and configure.ac Configuring libs/freetype2/ will create the wrapper Makefile handling all targets except 'all'. For 'make all' this will run the configure script of the distribution and then run 'make all' and usually also 'make install' to `install' the library and headers in the build tree 1.2.3. A mixed build system, e.g., for utils/xindy/ utils/xindy/xindy-2.4/ (almost) unmodified source tree utils/xindy/xindy-2.4-PATCHES/ modifications applied to that source tree utils/xindy/Makefile.am wrapper Makefile.am utils/xindy/configure.ac proxy configure.ac Configuring utils/xindy/ (or texk/lcdf-typetools/) will create the wrapper Makefile and the Makefiles of the distribution. The wrapper Makefile will essentially just invoke the top-level Makefile of the distribution. This requires that the Makefiles of the distribution allow a VPATH build, can handle all targets, and do not refer to $(top_srcdir) or $(top_builddir). 2. Running `configure' ====================== Running 'configure' for the top-level directory will in addition configure the directories auxdir/auxsub/, libs/, utils/, and texk/. This will determine the programs (utils/Util/ and texk/Prog/) to be built and libraries from the TL tree (texk/kpathsea/, texk/ptexenc/, and libs/Lib/) required by these programs. 3. Running `make' at the top-level ================================== First, if texk/kpathsea/ or texk/ptexenc/ is not yet configured, this runs `configure' and if required `make all' in that directory. Then this recurses into all required subdirectories texk/kpathsea/, texk/ptexenc/, libs/, utils/, and texk/ for the selected Make target: `default' or `all' to (re-)build, `check' to run tests, `install' etc. 3.1. Running `make' in libs/ ---------------------------- First, for each subdirectory libs/Lib/ not yet configured, this runs `configure' and if required `make all' in that directory. Then this recurses into all required subdirectories for the selected Make target: `default' or `all' to (re-)build, `check' to run tests, `install' etc. 3.2. Running `make' in utils/ and texk/ --------------------------------------- Quite similarly, for each subdirectory utils/Util/ and texk/Prog/ not yet configured, this first runs `configure' and if required `make all' in that directory. Subsequently, this recurses into all required subdirectories to (re-)build, run tests, install, etc. 3.3 Parallel builds ------------------- The TL build system allows for parallel builds ('make -j n' with n>1), carefully formulating dependencies as well as Make rules when a tool (such as 'tangle' or 'convert') creates several output files. This can considerably speed up a TL build on multi core systems. I usually use 'make j 4 -l 8.0' or even 'make -j' with up to 50 compilations in parallel. 3.4 Using a Configure Cache File -------------------------------- Further speed up of a TL build can be achieved by using a configure cache file, e.g., running 'configure -C' at the top level. 3.5 Comments ------------ 3.5.1 Comment 1 --------------- After successfully running 'make' from the top-level, as second 'make' has nothing to rebuild. When configure scripts or source files have been modified, a second 'make' will rebuild only as required. 3.5.2 Comment 2 --------------- If running 'make' from the top-level fails in a subdirectory libs/*/, utils/*/, or /texk/*/ you can remove that directory from the build tree, fix the problem, and once again run 'make' from the top-level. 3.5.3 Comment 3 --------------- The TL build system successfully runs 'make dist' and 'make distcheck' (at least on my {i686,x86_64}-linux-gnu systems), producing tarballs tex-live-20YY-MM-DD.tar.{gz,xz} from which everything can be (re-)built; please keep it that way! 3.5.4 Comment 4 --------------- The process described above configures all directories, even for libraries and programs not to be built. This is necessary for Make targets such as `dist' or `distcheck' that have to recurse into all subdirectories, whereas the normal targets `all', `check', or `install' only recurse into the required subdirectories. As a side effect, this allows building all programs `on demand'. If, e.g., building in utils/Util/ or texk/Prog/ has been disabled, the Makefile in that directory still contains all build rules. Thus running 'make' and 'make install' in that directory will build and install everything. Similarly, when, e.g., building e-TeX has been disabled (as by default), you can run 'make etex' (or 'make etex.exe') in the subdirectory texk/web2c/ to build e-TeX (although there is no simple way to have e-TeX installed). Consider the extreme case of running configure with '--disable-all-pkgs', i.e., no programs and consequently no libraries are to be built. Running 'make' will then configure all library and program directories but build nothing; similarly 'make install' will install nothing. Nevertheless next running, e.g., 'make tex' in the subdirectory texk/web2c/ will build first the Kpathsea library, then tangle, and finally tex.