Adding A New Tool

In general, the best way to learn how to write (code or even prose) is to read something similar. This principle applies to test cases and to test suites. Unfortunately, well-established test suites have a way of developing their own conventions: as test writers become more experienced with DejaGnu and with Tcl, they accumulate more utilities, and take advantage of more and more features of Expect and Tcl in general.

Inspecting such established test suites may make the prospect of creating an entirely new test suite appear overwhelming. Nevertheless, it is quite straightforward to get a new test suite going.

There is one test suite that is guaranteed not to grow more elaborate over time: both it and the tool it tests were created expressly to illustrate what it takes to get started with DejaGnu. The example/ directory of the DejaGnu distribution contains both an interactive tool called calc, and a test suite for it. Reading this test suite, and experimenting with it, is a good way to supplement the information in this section. (Thanks to Robert Lupton for creating calc and its test suite---and also the first version of this section of the manual!)

To help orient you further in this task, here is an outline of the steps to begin building a test suite for a program example.

• Create or select a directory to contain your new collection of tests. Change into that directory (shown here as testsuite):

  Create a configure.in file in this directory, to control configuration-dependent choices for your tests. So far as DejaGnu is concerned, the important thing is to set a value for the variable target_abbrev; this value is the link to the init file you will write soon. (For simplicity, we assume the environment is Unix, and use unix as the value.)

  What else is needed in configure.in depends on the requirements of your tool, your intended test environments, and which configure system you use. This example is a minimal configure.in for use with GNU Autoconf.

• Create Makefile.in (if you are using Autoconf), or Makefile.am(if you are using Automake), the source file used by configure to build your Makefile. If you are using GNU Automake.just add the keyword dejagnu to the AUTOMAKE_OPTIONS variable in your Makefile.am file. This will add all the Makefile support needed to run DejaGnu, and support the Make Check target.

  You also need to include two targets important to DejaGnu: check, to run the tests, and site.exp, to set up the Tcl copies of configuration-dependent values. This is called the Local Config File The check target must run the runtest program to execute the tests.

  The site.exp target should usually set up (among other things) the $tool variable for the name of your program. If the local site.exp file is setup correctly, it is possible to execute the tests by merely typing runtest on the command line.

  Example 1. Sample Makefile.in Fragment

  # Look for a local version of DejaGnu, otherwise use one in the path RUNTEST = `if test -f $(top_srcdir)/../dejagnu/runtest; then \ echo $(top_srcdir) ../dejagnu/runtest; \ else \ echo runtest; \ fi` # The flags to pass to runtest RUNTESTFLAGS = # Execute the tests check: site.exp all $(RUNTEST) $(RUNTESTFLAGS) \ --tool ${example} --srcdir $(srcdir) # Make the local config file site.exp: ./config.status Makefile @echo "Making a new config file..." -@rm -f ./tmp? @touch site.exp -@mv site.exp site.bak @echo "## these variables are automatically\ generated by make ##" > ./tmp0 @echo "# Do not edit here. If you wish to\ override these values" >> ./tmp0 @echo "# add them to the last section" >> ./tmp0 @echo "set host_os ${host_os}" >> ./tmp0 @echo "set host_alias ${host_alias}" >> ./tmp0 @echo "set host_cpu ${host_cpu}" >> ./tmp0 @echo "set host_vendor ${host_vendor}" >> ./tmp0 @echo "set target_os ${target_os}" >> ./tmp0 @echo "set target_alias ${target_alias}" >> ./tmp0 @echo "set target_cpu ${target_cpu}" >> ./tmp0 @echo "set target_vendor ${target_vendor}" >> ./tmp0 @echo "set host_triplet ${host_canonical}" >> ./tmp0 @echo "set target_triplet ${target_canonical}">>./tmp0 @echo "set tool binutils" >> ./tmp0 @echo "set srcdir ${srcdir}" >> ./tmp0 @echo "set objdir `pwd`" >> ./tmp0 @echo "set ${examplename} ${example}" >> ./tmp0 @echo "## All variables above are generated by\ configure. Do Not Edit ##" >> ./tmp0 @cat ./tmp0 > site.exp @sed < site.bak \ -e '1,/^## All variables above are.*##/ d' \ >> site.exp -@rm -f ./tmp?

• Create a directory (in testsuite) called config. Make a Tool Init File in this directory. Its name must start with the target_abbrev value, or be named default.exp so call it config/unix.exp for our Unix based example. This is the file that contains the target-dependent procedures. Fortunately, on Unix, most of them do not have to do very much in order for runtest to run.

  If the program being tested is not interactive, you can get away with this minimal unix.exp to begin with:

  Example 2. Simple Batch Program Tool Init File

  proc foo_exit {} {} proc foo_version {} {}

  If the program being tested is interactive, however, you might as well define a start routine and invoke it by using an init file like this:

  Example 3. Simple Interactive Program Tool Init File

  proc foo_exit {} {} proc foo_version {} {} proc foo_start {} { global ${examplename} spawn ${examplename} expect { -re "" {} } } # Start the program running we want to test foo_start

• Create a directory whose name begins with your tool's name, to contain tests. For example, if your tool's name is gcc, then the directories all need to start with "gcc.".

• Create a sample test file. Its name must end with .exp. You can use first-try.exp. To begin with, just write there a line of Tcl code to issue a message.

  Example 4. Testing A New Tool Config

  send_user "Testing: one, two...\n"

• Back in the testsuite (top level) directory, run configure. Typically you do this while in the build directory. You may have to specify more of a path, if a suitable configure is not available in your execution path.

• e now ready to triumphantly type make check or runtest. You should see something like this:

  Example 5. Example Test Case Run

  Test Run By rhl on Fri Jan 29 16:25:44 EST 1993 === example tests === Running ./example.0/first-try.exp ... Testing: one, two... === example Summary ===

  There is no output in the summary, because so far the example does not call any of the procedures that establish a test outcome.

• Write some real tests. For an interactive tool, you should probably write a real exit routine in fairly short order. In any case, you should also write a real version routine soon.