3 The focus of these tests is coverage of the code in scanner.mdc
4 with the aim of being able to detect regressions.
6 We have a few different input files, and scan each with a variety
7 of different setting and ensure the output is as expected.
9 The inputs are in sectoins called "test: name" and the expected
10 outputs are in "test: name,arg,arg,arg". As we cannot (yet)
11 extract a list of section names, we need to tell the make script
14 ###### File: scanner-tests.mk
19 tests:: scanner_test_suite
20 scanner_test_suite : md2c coverage_scanner
21 @rm -rf coverage; mkdir -p coverage
23 @for T in $(scanner_tests); do \
24 echo -n "Tests $$T ... "; \
25 i="$IFS"; IFS=,; set $$T; IFS="$$i"; t=$$1; shift; \
26 ./md2c scanner-tests.mdc "output: $$T" | grep -v '^#' > .tmp.want; \
27 echo '~~~~~~~' > .tmp.code ;\
28 ./md2c scanner-tests.mdc "test: $$t" | grep -v '^#' >> .tmp.code; \
29 ./coverage_scanner --file .tmp.code $${1+"$$@"} > .tmp.have; \
30 if ! cmp -s .tmp.want .tmp.have; then \
31 echo "FAILED"; diff -u .tmp.want .tmp.have; exit 1; fi ; \
36 @for i in coverage/#*.gcda; do mv $$i coverage/$${i##*#}; done
37 @gcov -o coverage scanner.c libscanner.c > /dev/null 2> /dev/null
38 @mv *.gcov coverage; [ -f .gcov ] && mv .gcov coverage || true
39 @awk '/NOTEST/ { next } /^ *[1-9]/ {ran+=1} /^ *###/ {skip+=1} \
40 END {printf "coverage: %6.2f%%\n", ran * 100 / (ran + skip); \
41 if (ran < (ran + skip) *0.94) exit(1) }' \
42 coverage/scanner.mdc.gcov
45 coverage_scanner: scanner.c libscanner.c libmdcode.o libnumber.o libstring.o
46 $(CC) $(CFLAGS) --coverage -fprofile-dir=coverage -o coverage_scanner \
47 scanner.c libscanner.c \
48 libmdcode.o libnumber.o libstring.o -licuuc -lgmp
52 Some simple tests... maybe all tests are simple.
53 Include a special test for numbers, as they are interesting.
56 scanner_tests += "test1,-r,if,then,+,-"
57 scanner_tests += "test1,-r,if,then,+,-,/"
58 scanner_tests += "test1,-r,--ignore-indent,if,then,+,-,/"
59 scanner_tests += "test1,-r,--ignore-indent,--ignore-newline,if,then,+,-,/"
60 scanner_tests += "test1,--ignore-indent,--ignore-newline,if,then,+,-,/"
61 scanner_tests += "test1,-Sz,--ignore-indent,--ignore-newline,if,then,+,-,/"
81 1234, 1.234 -123.456e45
83 "This is a string" &"so is this"a
89 I think # some comments are like this
90 and // some are like this
96 ###### output: test1,-r,if,then,+,-
148 18:0 number(1234) 1234
150 18:7 number(1.234) 617/500
152 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
154 19:0 number(0x1234) 4660
156 19:10 number(0x543p+3) 10776
158 20:0 string("This is a string") This is a string
160 20:20 string("so is this"a) a so is this
164 21:4 mstring("""\x0a This is a m..) bb This is a multi-\x0al..
168 26:8 lcomment(# some comments ar..)
171 27:4 lcomment(// some are like t..)
175 28:9 bcomment(/* can go\x0aover mul..)
183 ###### output: test1,-r,if,then,+,-,/
235 18:0 number(1234) 1234
237 18:7 number(1.234) 617/500
239 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
241 19:0 number(0x1234) 4660
243 19:10 number(0x543p+3) 10776
245 20:0 string("This is a string") This is a string
247 20:20 string("so is this"a) a so is this
251 21:4 mstring("""\x0a This is a m..) bb This is a multi-\x0al..
255 26:8 lcomment(# some comments ar..)
258 27:4 lcomment(// some are like t..)
262 28:9 bcomment(/* can go\x0aover mul..)
271 ###### output: test1,-r,--ignore-indent,if,then,+,-,/
311 18:0 number(1234) 1234
313 18:7 number(1.234) 617/500
315 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
317 19:0 number(0x1234) 4660
319 19:10 number(0x543p+3) 10776
321 20:0 string("This is a string") This is a string
323 20:20 string("so is this"a) a so is this
327 21:4 mstring("""\x0a This is a m..) bb This is a multi-\x0al..
331 26:8 lcomment(# some comments ar..)
334 27:4 lcomment(// some are like t..)
338 28:9 bcomment(/* can go\x0aover mul..)
347 ###### output: test1,-r,--ignore-indent,--ignore-newline,if,then,+,-,/
371 18:0 number(1234) 1234
373 18:7 number(1.234) 617/500
375 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
376 19:0 number(0x1234) 4660
378 19:10 number(0x543p+3) 10776
379 20:0 string("This is a string") This is a string
381 20:20 string("so is this"a) a so is this
384 21:4 mstring("""\x0a This is a m..) bb This is a multi-\x0al..
387 26:8 lcomment(# some comments ar..)
389 27:4 lcomment(// some are like t..)
392 28:9 bcomment(/* can go\x0aover mul..)
399 ###### output: test1,--ignore-indent,--ignore-newline,if,then,+,-,/
423 18:0 number(1234) 1234
425 18:7 number(1.234) 617/500
427 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
428 19:0 number(0x1234) 4660
430 19:10 number(0x543p+3) 10776
431 20:0 string("This is a string") This is a string
433 20:20 string("so is this"a) a so is this
436 21:4 mstring("""\x0a This is a m..) bb This is a multi-\x0al..
448 ###### output: test1,-Sz,--ignore-indent,--ignore-newline,if,then,+,-,/
472 18:0 number(1234) 1234
474 18:7 number(1.234) 617/500
476 18:14 number(123.456e45) 123456000000000000000000000000000000000000000000
477 19:0 number(0x1234) 4660
479 19:10 number(0x543p+3) 10776
520 scanner_tests += "testnum"
536 "Now for some non-number"
548 ###### output: testnum
550 2:0 number(12345) 12345
552 3:0 number(1234.56) 30864/25
554 4:0 number(1234.56e7) 12345600000
556 5:0 number(1234.56e-7) 1929/15625000
558 6:0 number(0x1234) 4660
560 7:0 number(0x123,456) 596523/2048
562 8:0 number(0o777) 511
564 9:0 number(0o111.111p4) 37449/32
566 10:0 number(0b11011110p3) 1776
569 12:0 number(123 456 789) 123456789
571 13:0 number(0x1234_5678_9abc) 20015998343868
574 15:0 string("Now for some non-..) Now for some non-n..
576 16:0 number(1234p4) BAD NUMBER
582 18:0 number(01234) BAD NUMBER
584 19:0 number(0c1234) BAD NUMBER
586 20:0 number(123.456e1a) a 30864/25
588 21:0 number(123.e4) 1230000
590 22:0 number(0x123 456) BAD NUMBER
592 23:0 number(0b1234) BAD NUMBER
594 24:0 number(123_345) 123345
606 Now to test for some errors ... though things I thought would be errors
610 scanner_tests += "errtest,-r,--ignore-ident,--ignore-mark,-W_,-w_,if,then,+,-"
611 scanner_tests += "errtest,-r,--ignore-ident,--ignore-mark,-N,if,then,+,-"
615 multiple decimal pointer 3.141.59
616 "Check for decimal commas" 3,14159 = 3,141,59
617 """A multi-string must not have text here
619 "or after close" + """
623 /* No embedded /* comments */ in comments */
625 * a multiline comment */ Error
627 " \\ \t \n special chars in strings"
629 ###### output: errtest,-r,--ignore-ident,--ignore-mark,-W_,-w_,if,then,+,-
635 2:25 number(3.141) 3141/1000
639 3:0 string("Check for decimal..) Check for decimal ..
640 3:27 number(3,14159) 314159/100000
642 3:37 number(3,141) 3141/1000
647 4:2 ERROR("A multi-string mu..)
649 5:0 ERROR("""\x0a"or after clos..)
653 10:0 ERROR(/* No embedded /*)
654 10:15 bcomment(/* comments */)
656 10:33 ERROR(comments)
659 11:0 ERROR(/* or content afte..)
663 14:0 string(" \\\\ \\t \\n specia..) \\ \x09 \x0a special ch..
667 ###### output: errtest,-r,--ignore-ident,--ignore-mark,-N,if,then,+,-
681 3:0 string("Check for decimal..) Check for decimal ..
700 4:2 ERROR("A multi-string mu..)
702 5:0 ERROR("""\x0a"or after clos..)
706 10:0 ERROR(/* No embedded /*)
707 10:15 bcomment(/* comments */)
709 10:33 ERROR(comments)
712 11:0 ERROR(/* or content afte..)
716 14:0 string(" \\\\ \\t \\n specia..) \\ \x09 \x0a special ch..
722 We need to test various aspects of tokenizing code that is stored
723 in multiple nodes. For example, comments and multi-line strings mustn't
724 cross a node boundary.
726 For this we tell `scanner` to extract sections directly from this file.
727 As the file changes, line numbers might change as well, so we need to factor
728 that out when testing. A simple awk script can normalise the first line number
732 @for T in $(scanner_section_tests); do \
733 echo -n "Test $$T ... "; \
734 i="$IFS"; IFS=,; set $$T; IFS="$$i"; section="$$1"; shift; \
735 ./md2c scanner-tests.mdc "output: $$T" | grep -v '^#' > .tmp.want; \
736 ./coverage_scanner --file scanner-tests.mdc --section "test: $$section" \
737 $${1+"$$@"} | awk -F: ' BEGIN {OFS=":"} $$1 ~ /^[0-9]/ {if (!first) first = $$1 - 1; \
738 $$1 = $$1 - first} { print } '> .tmp.have; \
739 if ! cmp -s .tmp.want .tmp.have; then \
740 echo "FAILED"; diff -u .tmp.want .tmp.have; exit 1; fi ; \
745 scanner_section_tests += section1 section_string section_comment
747 ###### test: section1
749 foreach s in sections:
758 ###### output: section1
759 Tokenizing: test: section1
781 ###### test: section_string
783 A sting mustn't cross
791 ###### output: section_string
792 Tokenizing: test: section_string
795 1:12 ERROR('''\x0a\x09 A sting mus..)
810 ###### test: section_comment
811 /* Mult-line comment must stay within
816 a single node, they cannot cross nodes.
818 ###### output: section_comment
819 Tokenizing: test: section_comment
820 1:8 ERROR(/* Mult-line comme..)
837 These tests test bugs that were found in practice, and so prevent them recuring.
839 The "bad_indent" test was written because I was seeing a TK_in before the
840 "program" instead of TK_newline
843 scanner_tests += "bad_indent"
845 ###### test: bad_indent
854 ###### output: bad_indent
868 4:23 string("string") string