PATCH: Test framework
The ngspice effort is hindered by a lack of a testing environment. I
have tried to remedy this situation somewhat. The attached patch
against ngspice-rework-10 introduces a way to confirm the output of a
testrun against previously generated output.
The testsuite is seriously lacking tests. I encourage everyone who
changes anything in the source code to write tests that support the
change. Preferrably you should write the tests in anticipation of the
source change.
o Simple model tests that can be easily verified
o Erroneous netlists
o Frontend behaviour and its error messages
o Add your own suggestions here...
I am also thinking about introducing a `--debug=DEBUGLEVEL' switch.
Depending on DEBUGLEVEL, spice can spit out a hoist of debugging
messages. This is useful for testing the inner workings of spice
without burdening the simulation when it is not needed.
Happy hacking,
--
Arno
diff -ruN ng-spice-rework-10.orig/configure.in ng-spice-tests/configure.in
--- ng-spice-rework-10.orig/configure.in Tue Apr 4 20:55:53 2000
+++ ng-spice-tests/configure.in Wed Apr 26 07:55:16 2000
@@ -242,8 +242,4 @@
src/misc/Makefile \
src/parser/Makefile \
tests/Makefile \
-tests/check.sh
)
-
-dnl this sucks but haven't found a better way
-AC_OUTPUT_COMMANDS(chmod a+x tests/check.sh)
diff -ruN ng-spice-rework-10.orig/tests/ChangeLog
ng-spice-tests/tests/ChangeLog
--- ng-spice-rework-10.orig/tests/ChangeLog Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/ChangeLog Thu Jan 1 01:00:00 1970
@@ -1,8 +0,0 @@
-1999-09-08 Emmanuel Rouat <emmanuel.rouat@wanadoo.fr>
-
- * check2.cir: added this test in 'make check'
-
-1999-09-07 Emmanuel Rouat <emmanuel.rouat@wanadoo.fr>
-
- * check.sh.in : added simple check ('make check')
-
diff -ruN ng-spice-rework-10.orig/tests/Makefile.am
ng-spice-tests/tests/Makefile.am
--- ng-spice-rework-10.orig/tests/Makefile.am Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/Makefile.am Wed Apr 26 07:55:45 2000
@@ -1,12 +1,18 @@
## Process this file with automake to produce Makefile.in
+TESTS = diffpair.sh fourbitadder.sh resistor.sh
+TESTS_ENVIRONMENT = $(SHELL)
-TESTS = check.sh
-
-
-EXTRA_DIST = README \
- check1.cir \
- check2.cir
+EXTRA_DIST = \
+ README \
+ config.sh \
+ $(TESTS) \
+ diffpair.cir \
+ diffpair.out \
+ fourbitadder.cir \
+ fourbitadder.out \
+ resistor.cir \
+ resistor.out
MAINTAINERCLEANFILES = Makefile.in
diff -ruN ng-spice-rework-10.orig/tests/README ng-spice-tests/tests/README
--- ng-spice-rework-10.orig/tests/README Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/README Wed Apr 26 00:16:36 2000
@@ -1,3 +1,5 @@
-TEST FILES FOR NG-SPICE
-
+TO ADD NEW TESTS
+Take an existing test and adopt it to your liking. Add the test
+script and its supporting files to Makefile.am. Use `make check' to
+see your new test in action.
diff -ruN ng-spice-rework-10.orig/tests/check.sh.in
ng-spice-tests/tests/check.sh.in
--- ng-spice-rework-10.orig/tests/check.sh.in Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/check.sh.in Thu Jan 1 01:00:00 1970
@@ -1,27 +0,0 @@
-#! /bin/sh
-# @configure_input@
-
-# This is a very basic test - it is only here to insure that
-# the ngspice binary built during a 'make distcheck' works at
-# least a little bit . We don't care about the result, we only
-# check that there is no segfault or whatever.
-# A much more aggressive test will be implemented later.
-
-
-PACKAGE=@PACKAGE@
-VERSION=@VERSION@
-srcdir=@srcdir@
-
-NGSPICE=../src/ngspice
-
-TESTFILE=$srcdir/check1.cir
-
-$NGSPICE < $TESTFILE
-
-TESTFILE=$srcdir/check2.cir
-
-$NGSPICE < $TESTFILE
-
-
-
-# more test files could be added here.
diff -ruN ng-spice-rework-10.orig/tests/check1.cir
ng-spice-tests/tests/check1.cir
--- ng-spice-rework-10.orig/tests/check1.cir Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/check1.cir Thu Jan 1 01:00:00 1970
@@ -1,30 +0,0 @@
-simple differential pair - CM and DM dc sensitivity
-
-* Models:
-.model qnl npn(bf=80 rb=100 ccs=2pf tf=0.3ns tr=6ns cje=3pf cjc=2pf va=50)
-.model qnr npn(bf=80 rb=100 ccs=2pf tf=0.3ns tr=6ns cje=3pf cjc=2pf va=50)
-
-* Circuit description:
-q1 4 2 6 qnr
-q2 5 3 6 qnl
-rs1 11 2 1k
-rs2 3 1 1k
-rc1 4 8 10k
-rc2 5 8 10k
-q3 7 7 9 qnl
-q4 6 7 9 qnr
-rbias 7 8 20k
-
-* Inputs/Supplies:
-vcm 1 0 dc 0 sin(0 0.1 5meg) ac 1
-vdm 1 11 dc 0 sin(0 0.1 5meg) ac 1
-vcc 8 0 12
-vee 9 0 -12
-
-* Analysys:
-.tf v(5) vcm
-.tf v(5) vdm
-.sens v(5,4)
-
-
-.end
\ No newline at end of file
diff -ruN ng-spice-rework-10.orig/tests/check2.cir
ng-spice-tests/tests/check2.cir
--- ng-spice-rework-10.orig/tests/check2.cir Sun Mar 26 12:30:52 2000
+++ ng-spice-tests/tests/check2.cir Thu Jan 1 01:00:00 1970
@@ -1,78 +0,0 @@
-4 bit adder
-
-* Models:
-.MODEL dmod D
-.MODEL qmod NPN(BF=75 RB=100 CJE=1PF CJC=3PF)
-
-
-.SUBCKT NAND 1 2 3 4
-* noeuds: entrees(2) sortie vcc
-q1 9 5 1 qmod
-d1clamp 0 1 dmod
-q2 9 5 2 qmod
-d2clamp 0 2 dmod
-rb 4 5 4k
-r1 4 6 1.6k
-q3 6 9 8 qmod
-r2 8 0 1k
-rc 4 7 130
-q4 7 6 10 qmod
-dvbedrop 10 3 dmod
-q5 3 8 0 qmod
-.ends NAND
-
-.SUBCKT ONEBIT 1 2 3 4 5 6
-* noeuds entrees(2) ,carryin, sortie, carryout, vcc
-x1 1 2 7 6 NAND
-x2 1 7 8 6 NAND
-x3 2 7 9 6 NAND
-x4 8 9 10 6 NAND
-x5 3 10 11 6 NAND
-x6 3 11 12 6 NAND
-x7 10 11 13 6 NAND
-x8 12 13 4 6 NAND
-x9 11 7 5 6 NAND
-.ends ONEBIT
-
-.SUBCKT TWOBIT 1 2 3 4 5 6 7 8 9
-* noeuds
-x1 1 2 7 5 10 9 ONEBIT
-x2 3 4 10 6 8 9 ONEBIT
-.ends TWOBIT
-
-.SUBCKT FOURBIT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-
-x1 1 2 3 4 9 10 13 16 15 TWOBIT
-x2 5 6 7 8 11 12 16 14 15 TWOBIT
-.ends FOURBIT
-
-
-
-* Inputs/Supplies:
-
-vcc 99 0 DC 5V
-VIN1A 1 0 DC 0 pulse(0 3 0 10ns 10ns 10ns 50ns)
-VIN1B 2 0 DC 0 pulse(0 3 0 10ns 10ns 20ns 100ns)
-VIN2A 3 0 DC 0 pulse(0 3 0 10ns 10ns 40ns 200ns)
-VIN2B 4 0 DC 0 pulse(0 3 0 10ns 10ns 80ns 400ns)
-VIN3A 5 0 DC 0 pulse(0 3 0 10ns 10ns 160ns 800ns)
-VIN3B 6 0 DC 0 pulse(0 3 0 10ns 10ns 320ns 1600ns)
-VIN4A 7 0 DC 0 pulse(0 3 0 10ns 10ns 640ns 3200ns)
-VIN4B 8 0 DC 0 pulse(0 3 0 10ns 10ns 1280ns 6400ns)
-
-* Circuit description:
-x1 1 2 3 4 5 6 7 8 9 10 11 12 0 13 99 FOURBIT
-rbit0 9 0 1k
-rbit1 10 0 1k
-rbit2 11 0 1k
-rbit3 12 0 1k
-rcout 13 0 1k
-
-* Analysys:
-.tran 1ns 6ns
-.print tran v(1)
-
-
-.end
-
-
diff -ruN ng-spice-rework-10.orig/tests/config.sh
ng-spice-tests/tests/config.sh
--- ng-spice-rework-10.orig/tests/config.sh Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/config.sh Wed Apr 26 07:59:54 2000
@@ -0,0 +1,13 @@
+#! /bin/sh
+
+NGSPICE=../src/ngspice
+DIFFPIPE="Analysis|CPU|memory|Date|Note"
+
+function spicetest () {
+ $NGSPICE < $srcdir/$1.cir 2>&1 | egrep -v $DIFFPIPE > $1.test
+ if diff -u $1.test $srcdir/$1.out; then
+ rm $1.test
+ exit 0
+ fi
+ exit 1
+}
diff -ruN ng-spice-rework-10.orig/tests/diffpair.cir
ng-spice-tests/tests/diffpair.cir
--- ng-spice-rework-10.orig/tests/diffpair.cir Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/diffpair.cir Wed Apr 26 00:04:17 2000
@@ -0,0 +1,29 @@
+simple differential pair - CM and DM dc sensitivity
+
+* Models:
+.model qnl npn(bf=80 rb=100 ccs=2pf tf=0.3ns tr=6ns cje=3pf cjc=2pf va=50)
+.model qnr npn(bf=80 rb=100 ccs=2pf tf=0.3ns tr=6ns cje=3pf cjc=2pf va=50)
+
+* Circuit description:
+q1 4 2 6 qnr
+q2 5 3 6 qnl
+rs1 11 2 1k
+rs2 3 1 1k
+rc1 4 8 10k
+rc2 5 8 10k
+q3 7 7 9 qnl
+q4 6 7 9 qnr
+rbias 7 8 20k
+
+* Inputs/Supplies:
+vcm 1 0 dc 0 sin(0 0.1 5meg) ac 1
+vdm 1 11 dc 0 sin(0 0.1 5meg) ac 1
+vcc 8 0 12
+vee 9 0 -12
+
+* Analysys:
+.tf v(5) vcm
+.tf v(5) vdm
+.sens v(5,4)
+
+.end
diff -ruN ng-spice-rework-10.orig/tests/diffpair.out
ng-spice-tests/tests/diffpair.out
--- ng-spice-rework-10.orig/tests/diffpair.out Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/diffpair.out Wed Apr 26 00:03:33 2000
@@ -0,0 +1,19 @@
+
+Circuit: simple differential pair - CM and DM dc sensitivity
+
+Circuit: simple differential pair - CM and DM dc sensitivity
+
+
+Transfer function information:
+transfer_function = -1.10340e-01
+output_impedance_at_v(5) = 9.446409e+03
+vcm#input_impedance = 1.792504e+06
+
+Transfer function information:
+transfer_function = -8.78993e+01
+output_impedance_at_v(5) = 9.446409e+03
+vdm#input_impedance = 8.934967e+03
+
+
+
+
diff -ruN ng-spice-rework-10.orig/tests/diffpair.sh
ng-spice-tests/tests/diffpair.sh
--- ng-spice-rework-10.orig/tests/diffpair.sh Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/diffpair.sh Wed Apr 26 16:51:05 2000
@@ -0,0 +1,4 @@
+#! /bin/sh
+
+. $srcdir/config.sh
+spicetest diffpair
diff -ruN ng-spice-rework-10.orig/tests/fourbitadder.cir
ng-spice-tests/tests/fourbitadder.cir
--- ng-spice-rework-10.orig/tests/fourbitadder.cir Thu Jan 1 01:00:00
1970
+++ ng-spice-tests/tests/fourbitadder.cir Tue Apr 25 23:56:13 2000
@@ -0,0 +1,78 @@
+4 bit adder
+
+* Models:
+.MODEL dmod D
+.MODEL qmod NPN(BF=75 RB=100 CJE=1PF CJC=3PF)
+
+
+.SUBCKT NAND 1 2 3 4
+* noeuds: entrees(2) sortie vcc
+q1 9 5 1 qmod
+d1clamp 0 1 dmod
+q2 9 5 2 qmod
+d2clamp 0 2 dmod
+rb 4 5 4k
+r1 4 6 1.6k
+q3 6 9 8 qmod
+r2 8 0 1k
+rc 4 7 130
+q4 7 6 10 qmod
+dvbedrop 10 3 dmod
+q5 3 8 0 qmod
+.ends NAND
+
+.SUBCKT ONEBIT 1 2 3 4 5 6
+* noeuds entrees(2) ,carryin, sortie, carryout, vcc
+x1 1 2 7 6 NAND
+x2 1 7 8 6 NAND
+x3 2 7 9 6 NAND
+x4 8 9 10 6 NAND
+x5 3 10 11 6 NAND
+x6 3 11 12 6 NAND
+x7 10 11 13 6 NAND
+x8 12 13 4 6 NAND
+x9 11 7 5 6 NAND
+.ends ONEBIT
+
+.SUBCKT TWOBIT 1 2 3 4 5 6 7 8 9
+* noeuds
+x1 1 2 7 5 10 9 ONEBIT
+x2 3 4 10 6 8 9 ONEBIT
+.ends TWOBIT
+
+.SUBCKT FOURBIT 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
+
+x1 1 2 3 4 9 10 13 16 15 TWOBIT
+x2 5 6 7 8 11 12 16 14 15 TWOBIT
+.ends FOURBIT
+
+
+
+* Inputs/Supplies:
+
+vcc 99 0 DC 5V
+VIN1A 1 0 DC 0 pulse(0 3 0 10ns 10ns 10ns 50ns)
+VIN1B 2 0 DC 0 pulse(0 3 0 10ns 10ns 20ns 100ns)
+VIN2A 3 0 DC 0 pulse(0 3 0 10ns 10ns 40ns 200ns)
+VIN2B 4 0 DC 0 pulse(0 3 0 10ns 10ns 80ns 400ns)
+VIN3A 5 0 DC 0 pulse(0 3 0 10ns 10ns 160ns 800ns)
+VIN3B 6 0 DC 0 pulse(0 3 0 10ns 10ns 320ns 1600ns)
+VIN4A 7 0 DC 0 pulse(0 3 0 10ns 10ns 640ns 3200ns)
+VIN4B 8 0 DC 0 pulse(0 3 0 10ns 10ns 1280ns 6400ns)
+
+* Circuit description:
+x1 1 2 3 4 5 6 7 8 9 10 11 12 0 13 99 FOURBIT
+rbit0 9 0 1k
+rbit1 10 0 1k
+rbit2 11 0 1k
+rbit3 12 0 1k
+rcout 13 0 1k
+
+* Analysys:
+.tran 1ns 6ns
+.print tran v(1)
+
+
+.end
+
+
diff -ruN ng-spice-rework-10.orig/tests/fourbitadder.out
ng-spice-tests/tests/fourbitadder.out
--- ng-spice-rework-10.orig/tests/fourbitadder.out Thu Jan 1 01:00:00
1970
+++ ng-spice-tests/tests/fourbitadder.out Wed Apr 26 00:03:36 2000
@@ -0,0 +1,77 @@
+
+Circuit: 4 bit adder
+
+Circuit: 4 bit adder
+
+
+ 4 bit adder
+--------------------------------------------------------------------------------
+Index time v(1)
+--------------------------------------------------------------------------------
+0 0.000000e+00 0.000000e+00
+1 1.200000e-12 3.600000e-04
+2 1.542463e-12 4.627390e-04
+3 2.227390e-12 6.682170e-04
+4 3.597243e-12 1.079173e-03
+5 6.336950e-12 1.901085e-03
+6 1.181636e-11 3.544909e-03
+7 2.277519e-11 6.832557e-03
+8 4.469284e-11 1.340785e-02
+9 8.852815e-11 2.655844e-02
+10 1.761988e-10 5.285963e-02
+11 2.824896e-10 8.474689e-02
+12 4.024896e-10 1.207469e-01
+13 5.224896e-10 1.567469e-01
+14 6.424896e-10 1.927469e-01
+15 7.624896e-10 2.287469e-01
+16 8.824896e-10 2.647469e-01
+17 1.002490e-09 3.007469e-01
+18 1.122490e-09 3.367469e-01
+19 1.242490e-09 3.727469e-01
+20 1.362490e-09 4.087469e-01
+21 1.482490e-09 4.447469e-01
+22 1.602490e-09 4.807469e-01
+23 1.722490e-09 5.167469e-01
+24 1.842490e-09 5.527469e-01
+25 1.962490e-09 5.887469e-01
+26 2.082490e-09 6.247469e-01
+27 2.202490e-09 6.607469e-01
+28 2.322490e-09 6.967469e-01
+29 2.442490e-09 7.327469e-01
+30 2.562490e-09 7.687469e-01
+31 2.682490e-09 8.047469e-01
+32 2.802490e-09 8.407469e-01
+33 2.922490e-09 8.767469e-01
+34 3.042490e-09 9.127469e-01
+35 3.162490e-09 9.487469e-01
+36 3.282490e-09 9.847469e-01
+37 3.402490e-09 1.020747e+00
+38 3.522490e-09 1.056747e+00
+39 3.642490e-09 1.092747e+00
+40 3.762490e-09 1.128747e+00
+41 3.882490e-09 1.164747e+00
+42 4.002490e-09 1.200747e+00
+43 4.122490e-09 1.236747e+00
+44 4.242490e-09 1.272747e+00
+45 4.362490e-09 1.308747e+00
+46 4.482490e-09 1.344747e+00
+47 4.602490e-09 1.380747e+00
+48 4.722490e-09 1.416747e+00
+49 4.842490e-09 1.452747e+00
+50 4.962490e-09 1.488747e+00
+51 5.082490e-09 1.524747e+00
+52 5.202490e-09 1.560747e+00
+53 5.322490e-09 1.596747e+00
+54 5.442490e-09 1.632747e+00
+
+Index time v(1)
+--------------------------------------------------------------------------------
+55 5.562490e-09 1.668747e+00
+56 5.682490e-09 1.704747e+00
+57 5.802490e-09 1.740747e+00
+58 5.922490e-09 1.776747e+00
+59 6.000000e-09 1.800000e+00
+
+
+
+
diff -ruN ng-spice-rework-10.orig/tests/fourbitadder.sh
ng-spice-tests/tests/fourbitadder.sh
--- ng-spice-rework-10.orig/tests/fourbitadder.sh Thu Jan 1 01:00:00
1970
+++ ng-spice-tests/tests/fourbitadder.sh Wed Apr 26 16:51:15 2000
@@ -0,0 +1,4 @@
+#! /bin/sh
+
+. $srcdir/config.sh
+spicetest fourbitadder
diff -ruN ng-spice-rework-10.orig/tests/resistor.cir
ng-spice-tests/tests/resistor.cir
--- ng-spice-rework-10.orig/tests/resistor.cir Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/resistor.cir Tue Apr 25 23:56:13 2000
@@ -0,0 +1,9 @@
+A simple resistor with a voltage source
+
+R1 1 0 10k
+V1 1 0 1
+
+.TRAN 1ns 6ns
+.PRINT TRAN I(V1)
+
+.END
diff -ruN ng-spice-rework-10.orig/tests/resistor.out
ng-spice-tests/tests/resistor.out
--- ng-spice-rework-10.orig/tests/resistor.out Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/resistor.out Wed Apr 26 00:03:37 2000
@@ -0,0 +1,75 @@
+
+Circuit: A simple resistor with a voltage source
+
+Circuit: A simple resistor with a voltage source
+
+
+ A simple resistor with a voltage source
+--------------------------------------------------------------------------------
+Index time v1#branch
+--------------------------------------------------------------------------------
+0 0.000000e+00 -1.000000e-04
+1 1.200000e-12 -1.000000e-04
+2 2.400000e-12 -1.000000e-04
+3 4.800000e-12 -1.000000e-04
+4 9.600000e-12 -1.000000e-04
+5 1.920000e-11 -1.000000e-04
+6 3.840000e-11 -1.000000e-04
+7 7.680000e-11 -1.000000e-04
+8 1.536000e-10 -1.000000e-04
+9 2.736000e-10 -1.000000e-04
+10 3.936000e-10 -1.000000e-04
+11 5.136000e-10 -1.000000e-04
+12 6.336000e-10 -1.000000e-04
+13 7.536000e-10 -1.000000e-04
+14 8.736000e-10 -1.000000e-04
+15 9.936000e-10 -1.000000e-04
+16 1.113600e-09 -1.000000e-04
+17 1.233600e-09 -1.000000e-04
+18 1.353600e-09 -1.000000e-04
+19 1.473600e-09 -1.000000e-04
+20 1.593600e-09 -1.000000e-04
+21 1.713600e-09 -1.000000e-04
+22 1.833600e-09 -1.000000e-04
+23 1.953600e-09 -1.000000e-04
+24 2.073600e-09 -1.000000e-04
+25 2.193600e-09 -1.000000e-04
+26 2.313600e-09 -1.000000e-04
+27 2.433600e-09 -1.000000e-04
+28 2.553600e-09 -1.000000e-04
+29 2.673600e-09 -1.000000e-04
+30 2.793600e-09 -1.000000e-04
+31 2.913600e-09 -1.000000e-04
+32 3.033600e-09 -1.000000e-04
+33 3.153600e-09 -1.000000e-04
+34 3.273600e-09 -1.000000e-04
+35 3.393600e-09 -1.000000e-04
+36 3.513600e-09 -1.000000e-04
+37 3.633600e-09 -1.000000e-04
+38 3.753600e-09 -1.000000e-04
+39 3.873600e-09 -1.000000e-04
+40 3.993600e-09 -1.000000e-04
+41 4.113600e-09 -1.000000e-04
+42 4.233600e-09 -1.000000e-04
+43 4.353600e-09 -1.000000e-04
+44 4.473600e-09 -1.000000e-04
+45 4.593600e-09 -1.000000e-04
+46 4.713600e-09 -1.000000e-04
+47 4.833600e-09 -1.000000e-04
+48 4.953600e-09 -1.000000e-04
+49 5.073600e-09 -1.000000e-04
+50 5.193600e-09 -1.000000e-04
+51 5.313600e-09 -1.000000e-04
+52 5.433600e-09 -1.000000e-04
+53 5.553600e-09 -1.000000e-04
+54 5.673600e-09 -1.000000e-04
+
+Index time v1#branch
+--------------------------------------------------------------------------------
+55 5.793600e-09 -1.000000e-04
+56 5.913600e-09 -1.000000e-04
+57 6.000000e-09 -1.000000e-04
+
+
+
+
diff -ruN ng-spice-rework-10.orig/tests/resistor.sh
ng-spice-tests/tests/resistor.sh
--- ng-spice-rework-10.orig/tests/resistor.sh Thu Jan 1 01:00:00 1970
+++ ng-spice-tests/tests/resistor.sh Wed Apr 26 16:51:22 2000
@@ -0,0 +1,4 @@
+#! /bin/sh
+
+. $srcdir/config.sh
+spicetest resistor
PGP signature
Partial thread listing: