Announcing Dolt, a drop-in Libtool replacement which cuts build times in

---

Many packages use GNU autotools (automake and autoconf) to build, tothe point that "./configure && make" represents one of the most commonbuild procedures for Free Software packages.  Libraries usingautotools typically use GNU Libtool, partly because it works on almostany system and partly because autotools makes it difficult to dootherwise.  Packages which use these libraries sometimes use libtoolas well.Yet for many of these libraries and other packages, more than half ofthe build time goes into running the libtool shell script.Libtool knows how to handle libraries for umpteen different systems,including many ancient systems that have terrible shared librarysupport.  It has some extensive shell script logic to figure out howto build libraries for your system, and how to compile objects that goin those libraries.  This logic does an amazingly impressive job ofcoping with adverse conditions.  However, this logic all lives in an~8500 line, ~250kB shell script, which runs *every single time youcompile a source file*.This does not do wonders for performance.Meanwhile, modern systems such as GNU/Linux have reasonable librarymechanisms, and need relatively little of the machinery in libtool.On these common systems, it would significantly improve build times toavoid running that libtool machinery for every compilation.Thus, I wrote Dolt, a drop-in replacement for libtool's compilationmode.  Dolt runs any necessary system-specific orconfiguration-specific logic as part of configure, writes out a simpleshell script "doltcompile"[1], and substitutes it for libtool in theautomake variables LTCOMPILE and LTCXXCOMPILE.  If you use automake,autoconf, and libtool, then using Dolt just requires two steps:1) add "DOLT" after the call to LT_INIT, AC_PATH_LIBTOOL, or   AM_PATH_LIBTOOL in your configure.ac or configure.in script, and2) append dolt.m4 to your project's acinclude.m4.For any system Dolt does not support, it will transparently fall backto libtool.dolt.m4 takes up less than 4kB; it writes out a minimal doltcompilescript which never forks except to run the compiler or to mkdir the.libs directory if it doesn't already exist.  I have tested it withvarious projects, and benchmarked[2] its performance against the sameprojects using only libtool.  Results:kdelibs without dolt: 8m6.115skdelibs with dolt:    3m50.065sgtk+-2.0 without dolt: 2m31.825sgtk+-2.0 with dolt:    1m33.858slibx11 without dolt: 1m50.163slibx11 with dolt:    0m53.417slibxml2 without dolt: 0m25.722slibxml2 with dolt:    0m19.576sdbus without dolt: 0m20.062sdbus with dolt:    0m8.940sI have attached a snapshot of dolt.m4 for convenience.  You can alsoobtain the current version of Dolt from Git with:    git clone git://svcs.cs.pdx.edu/git/doltor download a snapshot tarball from<http://svcs.cs.pdx.edu/gitweb?p=dolt.git;a=snapshot;h=master;sf=tgz>.Please try Dolt with your project, and see if you get comparableperformance improvements.  If you want to make Dolt replace libtool onyour system, feel free to send me a patch to dolt.m4; just rememberthe basic tenet that any logic must run at configure time, not buildtime.  You can figure out what compiler flags libtool uses by running"touch dummy.c && libtool --mode=compile gcc -c dummy.c -o dummy.lo";that will print two compiler command lines, one for the shared objectand one for the static object.Future directions:* Support GNU/Linux on architectures other than x86 and x86-64.  I  think most will work with exactly the same compiler flags, but I  didn't want to add any architecture I couldn't test.* Support other systems.* Possibly try to run libtool on a dummy source file at configure time  to figure out the necessary flags to use when building library  objects, but that seems error-prone.* Replace libtool --mode=link.* Replace libtool --mode=install.* Optionally stop installing .la files.* Make dolt.m4's output of doltcompile cleaner.- Josh Triplett[1] "doltcompile" stands for "do ltcompile"; the alternate reading"dolt compile" led to the name "dolt".[2] General testing methodology:* Run ./configure && make && make clean, to make sure it builds and to get  everything cached.* Get the "before" time: time make >/dev/null 2>&1* Remove and re-extract the source.* Add dolt.m4 to acinclude.m4 and add DOLT to configure.in or configure.ac.* autoreconf -v -f -i && ./configure && make && make clean, to  make sure it still builds and to get everything cached again.* Get the "after" time: time make >/dev/null 2>&1

dnl dolt, a replacement for libtooldnl Copyright © 2007-2008 Josh Triplett <address@hidden>dnl Copying and distribution of this file, with or without modification,dnl are permitted in any medium without royalty provided the copyrightdnl notice and this notice are preserved.dnldnl To use dolt, invoke the DOLT macro immediately after the libtool macros.dnl Optionally, copy this file into acinclude.m4, to avoid the need to have itdnl installed when running autoconf on your project.AC_DEFUN([DOLT], [AC_REQUIRE([AC_CANONICAL_HOST])# dolt, a replacement for libtool# Josh Triplett <address@hidden>AC_PATH_PROG(DOLT_BASH, bash)AC_MSG_CHECKING([if libtool sucks])AC_MSG_RESULT([yup, it does])AC_MSG_CHECKING([if dolt supports this host])dolt_supported=yesif test x$DOLT_BASH = x; then    dolt_supported=nofiif test x$GCC != xyes; then    dolt_supported=noficase $host ini?86-*-linux*|x86_64-*-linux*) ;;*) dolt_supported=no ;;esacif test x$dolt_supported = xno ; then    AC_MSG_RESULT([no, falling back to libtool])else    AC_MSG_RESULT([yes, replacing libtool])dnl Start writing out doltcompile.    cat <<__DOLTCOMPILE__EOF__ >doltcompile#!$DOLT_BASH__DOLTCOMPILE__EOF__    cat <<'__DOLTCOMPILE__EOF__' >>doltcompileargs=("$[]@")for ((arg=0; arg<${#args@<:@@@:>@}; arg++)) ; do    if test x"${args@<:@$arg@:>@}" = x-o ; then        objarg=$((arg+1))        break    fidoneif test x$objarg = x ; then    echo 'Error: no -o on compiler command line' 1>&2    exit 1filo="${args@<:@$objarg@:>@}"obj="${lo%.lo}"if test x"$lo" = x"$obj" ; then    echo "Error: libtool object file name \"$lo\" does not end in .lo" 1>&2    exit 1fiobjbase="${obj##*/}"__DOLTCOMPILE__EOF__dnl Write out shared compilation code.    if test x$enable_shared = xyes; then        cat <<'__DOLTCOMPILE__EOF__' >>doltcompilelibobjdir="${obj%$objbase}.libs"if test ! -d "$libobjdir" ; then    mkdir "$libobjdir"    mkdir_ret=$?    if test "$mkdir_ret" -ne 0 && test ! -d "$libobjdir" ; then        exit $mkdir_ret    fifipic_object="$libobjdir/$objbase.o"args@<:@$objarg@:>@="$pic_object""${args@<:@@@:>@}" -fPIC -DPIC__DOLTCOMPILE__EOF__    fidnl Write out static compilation code.dnl Avoid duplicate compiler output if also building shared objects.    if test x$enable_static = xyes; then        cat <<'__DOLTCOMPILE__EOF__' >>doltcompilenon_pic_object="$obj.o"args@<:@$objarg@:>@="$non_pic_object"__DOLTCOMPILE__EOF__        if test x$enable_shared = xyes; then            cat <<'__DOLTCOMPILE__EOF__' >>doltcompile"${args@<:@@@:>@}" >/dev/null 2>&1__DOLTCOMPILE__EOF__        else            cat <<'__DOLTCOMPILE__EOF__' >>doltcompile"${args@<:@@@:>@}"__DOLTCOMPILE__EOF__        fi    fidnl Write out the code to write the .lo file.dnl The second line of the .lo file must match "^# Generated by .*libtool"    cat <<'__DOLTCOMPILE__EOF__' >>doltcompile{echo "# $lo - a libtool object file"echo "# Generated by doltcompile, not libtool"__DOLTCOMPILE__EOF__    if test x$enable_shared = xyes; then        cat <<'__DOLTCOMPILE__EOF__' >>doltcompileecho "pic_object='$pic_object'"__DOLTCOMPILE__EOF__    else        cat <<'__DOLTCOMPILE__EOF__' >>doltcompileecho pic_object=none__DOLTCOMPILE__EOF__    fi    if test x$enable_static = xyes; then        cat <<'__DOLTCOMPILE__EOF__' >>doltcompileecho "non_pic_object='$non_pic_object'"__DOLTCOMPILE__EOF__    else        cat <<'__DOLTCOMPILE__EOF__' >>doltcompileecho non_pic_object=none__DOLTCOMPILE__EOF__    fi    cat <<'__DOLTCOMPILE__EOF__' >>doltcompile} > "$lo"__DOLTCOMPILE__EOF__dnl Done writing out doltcompile; substitute it for libtool compilation.    chmod +x doltcompile    LTCOMPILE='$(top_builddir)/doltcompile $(COMPILE)'    AC_SUBST(LTCOMPILE)    LTCXXCOMPILE='$(top_builddir)/doltcompile $(CXXCOMPILE)'    AC_SUBST(LTCXXCOMPILE)fi# end dolt])