LinuxBernsteinMakeDebugSeries
From Wiki
This is the support page for Rocky Bernstein's Remake Debugger tutorials.
Contents |
Transcript
Hello - my name is Rocky Bernstein and I'd like to give a brief demo of some extensions to GNU Make that I have added to allow better error messages, allow tracing, and allow debugging Makefiles.
In a number of my GPL projects, I use GNU automake and GNU autoconf. When something went wrong with these, I wanted a way to understand what was going on and how those auto-programs converted what I wrote into the resulting configuration code. So I developed a GNU Make debugger and its companion, the bash debugger. Both of these can be found at http://bashdb.sf.net
"basic" tracing
GNU Make stores a great deal of information, but it is a challenge to figure out what information to present. Let me give an example of what I mean. Let's start out with something most beginners to GNU Make are tempted to do, but it doesn't do what I think most people want. Here's the most trivial Makefile [hello.Makefile]
all:
echo Hello World
If you get the "help usage" for Make:
make --help | head
Usage: make [options] [target] ...
Options:
-b, -m Ignored for compatibility.
-B, --always-make Unconditionally make all targets.
-C DIRECTORY, --directory=DIRECTORY
Change to DIRECTORY before doing anything.
-d Print lots of debugging information.
--debug[=FLAGS] Print various types of debugging information.
-e, --environment-overrides
Environment variables override makefiles.
you may be tempted to try the option "-d", debugging information. So let's try that:
make -d -f hello.Makefile GNU Make 3.80 Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Reading makefiles... Updating makefiles.... Considering target file `GNUmakefile'. File `GNUmakefile' does not exist. Looking for an implicit rule for `GNUmakefile'. Trying pattern rule with stem `GNUmakefile'. Trying implicit prerequisite `GNUmakefile.o'. ...
This trivial two-line Makefile produces just under 400 lines! Is it all that interesting? I don't think so - most of it seems to be a fascination with "implicit rule selection." And of this most of it is negative information, that is, patterns that aren't used.
A better option in GNU Make that I think most people want when they stumble over that -d is something called "basic" tracing. Here's a run of that
make --debug=basic -f hello.Makefile GNU Make 3.80 Copyright (C) 2002 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Reading makefiles... File `all' does not exist. Must remake target `all'. echo "Hello, World!" Hello, World! Successfully remade target file `all'.
Now let's put a little error into this file and we'll make it just a little more complicated. [hello2.Makefile]. Let's run that.
make -f hello2.Makefile "Hello, World!" /bin/sh: Hello, World!: command not found make: *** [hello] Error 127
We get an error. No surprise. But where were we?
Now let's compare this with the error output in the version of GNU Make called "remake" that supports better error reporting and debugging. [M-x compile]
remake -f hello.Makefile
(I'm doing this in a different [Emacs] buffer.)
make -f hello2.Makefile
"Hello, World!"
/bin/sh: Hello, World!: command not found
hello2.Makefile:4: *** [hello] Error 127
#0 hello at /home/rocky/showmedo/remake/hello2.Makefile:4
#1 all at /home/rocky/showmedo/remake/hello2.Makefile:1
Command-line arguments:
"-f hello2.Makefile"
They look almost the same. There are however some interesting subtle differences. Here we put in the name of the file, and the line number of the file. I ran this inside an Emacs compilation buffer, and the format of that line number is such that Emacs can position me inside the file on command. [The emacs keystroke is C-c C-c or compile-goto-error].
In this case figuring out the file name is easy, but GNU make can invoke or include other Makefiles and can get called recursively. We'll see that later on.
With position information, we can arrange front-ends to automatically position us at this point. And we have done so too in the Emacs and ddd frontends.
The other thing here is that we can see what was going on at the time of the error. So we were in line 1 at the all target and then we went to line 4 the hello target.
The last thing we do before exiting is give the command-line invocation. In this particular case it is not difficult to figure out what you typed, but remember that Makefiles can be recursive.
An automake example
Probably I haven't convinced you. So let's throw in automake and you'll see how quickly things can get complicated on the simplest automake Makefile.
Here we go through an extremely simple example. We are going to use autotools (autoconf, automake) on a hypothetical project. The configure script (configure from configure.ac) will figure out the name of a shell to use (@SHELL_PROG@) and substitute the value discovered into parrot.in to get a script called parrot.
Okay, let's get to work. Here's [parrot.in]:
#!@SHELL_PROG@ echo $*
There will be one test program in a test directory which runs parrot. Regression tests are a really good thing to have in a project. Here's the one-line initial version of the test program
[test/test.sh]
./parrot "Don't panic!"
Finally, the (initial) two-line automake [Makefile.am]
bin_SCRIPTS = parrot TESTS = test/test.sh
After creating configure and a Makefile using autoreconf -i -f,
wc -l Makefile configure
we get roughly a 500-line Makefile and a 2,700-line configure program.
Okay we are now ready for everything. We run the generated configuration script:
$ ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for ksh... /usr/bin/ksh configure: creating ./config.status config.status: creating Makefile config.status: creating parrot
make, make test**, and make dist work as expected. But how about make distcheck? The distcheck target should create the distribution file, unpack it, build it, run the regression test, install it, and finally remove it, checking to see that everything is removed. Let's try that.
$ make distcheck
{ test ! -d showmedo-1.0 || \
{ find showmedo-1.0 -type d ! -perm -200 -exec chmod u+w {} ';' \
&& rm -fr showmedo-1.0; }; }
mkdir showmedo-1.0
...
make check-TESTS
make[2]: Entering directory
`/home/rocky/showmedo/remake/automake/showmedo-1.0/_build'
make[2]: *** No rule to make target `test/test.sh', needed by `check-TESTS'. Stop.
make[2]: Leaving directory `/home/rocky/showmedo/remake/automake/showmedo-1.0/_build'
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/home/rocky/showmedo/remake/automake/showmedo-1.0/_build'
make: *** [distcheck] Error 2
This one is a little harder to pinpoint using GNU Make. Again using remake (step #2 under troubleshooting) gives some information presented previously but I think in a more helpful fashion. And of course it also gives line numbers.
What happened? Again reading bottom up, it seems that distcheck issued a recursive all to make check inside the showmedo-1.0/_build directory. As before, make check then needed to run target check-am which issued a 3rd make to run check-TESTS. Finally that check-TESTS target needed but couldn't find test/test.sh. A directory listing shows what we have:
ls showmedo-1.0/_build Makefile config.log config.status parrot
and we find there indeed is no test/test.sh. It didn't get added to the distribution file. At this point one could go scurrying to the automake documentation to glean what's wrong and that's not a bad approach.
However let us try tracing the dist target in more detail. This time we'll use remake's POSIX-shell line tracing. That is option -x or --trace.
This was listed as step #2 in the troubleshooting section. We'll switch back to the dist target to examine why test/test.sh didn't get added.
% remake -x dist Reading makefiles... ...
The "plus" lines after
##<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
and before any
##>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
are the output from running the shell (bash, ksh, sh) using "set -x" tracing, and that is why lines start out with +. The lines immediately before:
##<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
show the target and commands GNU Makefile.
So, let's look at the distdir target.
##>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
/home/rocky/showmedo/Makefile:262: distdir
{ test ! -d showmedo-1.0 || { find showmedo-1.0 -type d ! -perm -200 -exec
chmod u+w {} ';' && rm -fr showmedo-1.0; }; }
##<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
+ test '!' -d showmedo-1.0
+ find showmedo-1.0 -type d ! -perm -200 -exec chmod u+w {} ';'
+ rm -fr showmedo-1.0
##>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
we find that the "or" (||) part and "and" part (&&) were run.
Let us investigate a little further down:
##>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> /home/rocky/showmedo/Makefile:262: distdir srcdirstrip=`echo "." | sed 's|.|.|g'`; \ topsrcdirstrip=`echo "." | sed 's|.|.|g'`; \ list='README ./configure.ac ./aclocal.m4 ./Makefile.am ./Makefile.in ./ parrot.in ./configure AUTHORS COPYING ChangeLog INSTALL NEWS install-sh missing '; for file in $list; do .
which seem to be the target lists all of the files in the distribution. We see that test/test.sh indeed is not in the list. Looking in Makefile around line 265. (actually it's line 267) we see the variable controlling this is $(DISTFILES).
This completes part 1 of this demo. In part 2, we will show using the debugger perhaps using this error as a starting point.
Thank you.
Notes
- Although I show "make test" in the video, what should have been run is "make check". The "make test" command just checks that the test directory is around; it does not run the test. Still, were "make check" run, it would have shown that the test succeeds.
Fragmentary stuff that might be used in another demo
The following GNU Make-related question came up in another project http://lists.gnu.org/archive/html/libcdio-devel/2007-08/msg00001.html
I'm trying to compile libcdio-0.78.2 with MinGW. Autoconf runs fine, but when I run make, I get this: _cdio_generic.c:45:25: cdio/sector.h: No such file or directory followed by a few other "No such file" errors for a few other headers, and then a load of errors which I guess are the result of the unfound headers. I have those header files in the proper location, but why doesn't make find them? Anyone have any ideas?
In the session below, I first removed the object I wanted to have make rebuild. I then ran the GNU make debugger to continueup to that target I'm interested in. (One could also set a breakpoint or next until you get to the right target)
Then I get the command that's about to be run and examine variables which make up that command and the part of that command that contains the includes flags that are needed here. I selected variable INCLUDE because I happen to suspect that this is where to look. But if I didn't know, I'd just print or expand the variables one at a time.
remake -X _cdio_generic.o
Reading makefiles...
(:0):
expr 8 - 1
Updating makefiles....
(libcdio/lib/driver/Makefile:426)
Makefile.in:
mdb<0> c _cdio_generic.o
Updating goal targets....
File `_cdio_generic.o' does not exist.
Must remake target `_cdio_generic.o'.
(/src/external-cvs/libcdio/lib/driver/Makefile:524)
_cdio_generic.o: _cdio_generic.c _cdio_generic.c ../../config.h ...
mdb<1> t @ commands
_cdio_generic.o:
# commands to execute (from `Makefile', line 525):
if $(COMPILE) -MT $@ -MD -MP -MF "$(DEPDIR)/$*.Tpo" -c -o $@ $<; \
then mv -f "$(DEPDIR)/$*.Tpo" "$(DEPDIR)/$*.Po"; else rm -f
"$(DEPDIR)/$*.Tpo"; exit 1; fi
mdb<2> p COMPILE
Makefile:116 (origin: makefile) COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES)
$(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)
mdb<3> p INCLUDES
Makefile:360 (origin: makefile) INCLUDES = $(LIBCDIO_CFLAGS)
mdb<4> p LIBCDIO_CFLAGS
Makefile:217 (origin: makefile) LIBCDIO_CFLAGS = -I$(top_srcdir)/lib/driver
-I$(top_srcdir)/include/
mdb<5> p top_srcdir
Makefile:64 (origin: makefile) top_srcdir = ../..
