scanner: fix bug with indent at start of node.

If we find an indent, we assume there are delayed newlines
to comsume.
This is often true, but not at the start of a node.
So don't decrement delayed_lines if it is already zero.

Add the test case that found this bug for me.

Signed-off-by: NeilBrown <neil@brown.name>

diff --git a/csrc/scanner-tests.mdc b/csrc/scanner-tests.mdc
index 95c3556d2a8b01c1a22cf014fe191da23731ea7a..14a70572a6646886c485423fc770ffd452049b0e 100644 (file)
--- a/csrc/scanner-tests.mdc
+++ b/csrc/scanner-tests.mdc
@@ -714,3 +714,58 @@ sometimes aren't.
        14:0 string("  \\\\ \\t \\n specia..)    \\ \x09 \x0a special ch..
        15:0 newline()
        15:0 eof()
+
+## Ad-hoc test
+
+These tests test bugs that were found in practice, and so prevent them recuring.
+
+The "bad_indent" test was written because I was seeing a TK_in before the
+"program" instead of TK_newline
+
+###### test list
+       scanner_tests += "bad_indent"
+
+###### test: bad_indent
+
+               const:
+                       foo : number = 45
+                       bar := "string"
+               program:
+                       foo := 4
+                       print foo, bar
+
+###### output: bad_indent
+       Tokenizing: 
+       2:8 in()
+       2:8 ident(const)
+       2:13 mark(:)
+       3:16 in()
+       3:16 ident(foo)
+       3:20 mark(:)
+       3:22 ident(number)
+       3:29 mark(=)
+       3:31 number(45)  45
+       4:16 newline()
+       4:16 ident(bar)
+       4:20 mark(:=)
+       4:23 string("string")  string
+       5:8 newline()
+       5:8 out()
+       5:8 newline()
+       5:8 ident(program)
+       5:15 mark(:)
+       6:16 in()
+       6:16 ident(foo)
+       6:20 mark(:=)
+       6:23 number(4)  4
+       7:16 newline()
+       7:16 ident(print)
+       7:22 ident(foo)
+       7:25 mark(,)
+       7:27 ident(bar)
+       8:0 newline()
+       8:0 out()
+       8:0 newline()
+       8:0 out()
+       8:0 newline()
+       8:0 eof()

diff --git a/csrc/scanner.mdc b/csrc/scanner.mdc
index 21ec828c1d36dea273cabe05c3a3dce3844c29a2..80adb2a3ad1f7b1684af3ee7fb1f77006492c7a0 100644 (file)
--- a/csrc/scanner.mdc
+++ b/csrc/scanner.mdc
@@ -854,7 +854,8 @@ information and return one token.
                    state->indent_level < sizeof(state->indent_sizes)-1) {
                        state->indent_level += 1;
                        state->indent_sizes[state->indent_level] = state_indent(state);
-                       state->delayed_lines -= 1;
+                       if (state->delayed_lines)
+                               state->delayed_lines -= 1;
                        tk.num = TK_in;
                        return tk;
                }