oceani-tests: add tests for error cases.

Only one test so far, but this infrastructure will be useful
for others.

Signed-off-by: NeilBrown <neil@brown.name>

diff --git a/csrc/oceani-tests.mdc b/csrc/oceani-tests.mdc
index 78986fcb67a47a8b6e903317618ab332f72f1681..80160a0becc355a704e9d99157d563e2daba8675 100644 (file)
--- a/csrc/oceani-tests.mdc
+++ b/csrc/oceani-tests.mdc
@@ -393,6 +393,43 @@ Here I break it into two parts, keeping the array code separate.
        list[ 53 ]= 9040
        list[ 54 ]= 9768
 
+## Test code with syntax errors
+
+Syntax errors aren't handled well yet - the result is almost always a
+single message about the first error.  So this section will be fairly
+thin until we add proper parsing recovery in the face of common errors.
+
+To handle erronous code, we need a different set of tests, as we need to
+capture `stderr`. The same test code will be used for type errors too.
+As error messages contain the line number, and we don't want changes to
+this file to change the reported numbers, we copy the code into a
+separate file first, then run from there.
+
+###### test code
+       @for t in $(oceani_failing_tests); do \
+           echo -n "Test $$t ... "; \
+           ./md2c oceani-tests.mdc "output: $$t" | grep -v '^#' > .tmp.want; \
+           echo '``````' > .tmp.code; \
+           ./md2c oceani-tests.mdc "test: $$t" | grep -v '^#' >> .tmp.code; \
+           ./oceani .tmp.code > .tmp.have 2>&1; \
+           if ! cmp -s .tmp.want .tmp.have; then \
+              echo "FAILED"; diff -u .tmp.want .tmp.have ; exit 1; fi ;\
+           echo  "passed"; \
+           ./coverage_oceani --section "test: $$t" oceani-tests.mdc > /dev/null 2>&1 ;\
+       done || true
+
+###### test list
+       oceani_failing_tests := syn1
+
+###### test: syn1
+
+       program:
+               if then else while do
+
+###### output: syn1
+       .tmp.code:3:11: error: unhandled parse error: then
+
+
 ## Test erroneous command line args
 
 To improve coverage, we want to test correct handling of strange command