From: NeilBrown <neil@brown.name>
Date: Sat, 8 Jun 2019 10:18:35 +0000 (+1000)
Subject: scanner: fix bug with indent at start of node.
X-Git-Url: https://ocean-lang.org/code/?p=ocean;a=commitdiff_plain;h=19784ed2b95e5b5241207361eea0ff510ff416d7

scanner: fix bug with indent at start of node.

If we find an indent, we assume there are delayed newlines
to comsume.
This is often true, but not at the start of a node.
So don't decrement delayed_lines if it is already zero.

Add the test case that found this bug for me.

Signed-off-by: NeilBrown <neil@brown.name>
---

diff --git a/csrc/scanner-tests.mdc b/csrc/scanner-tests.mdc
index 95c3556..14a7057 100644
--- a/csrc/scanner-tests.mdc
+++ b/csrc/scanner-tests.mdc
@@ -714,3 +714,58 @@ sometimes aren't.
 	14:0 string("  \\\\ \\t \\n specia..)    \\ \x09 \x0a special ch..
 	15:0 newline()
 	15:0 eof()
+
+## Ad-hoc test
+
+These tests test bugs that were found in practice, and so prevent them recuring.
+
+The "bad_indent" test was written because I was seeing a TK_in before the
+"program" instead of TK_newline
+
+###### test list
+	scanner_tests += "bad_indent"
+
+###### test: bad_indent
+
+		const:
+			foo : number = 45
+			bar := "string"
+		program:
+			foo := 4
+			print foo, bar
+
+###### output: bad_indent
+	Tokenizing: 
+	2:8 in()
+	2:8 ident(const)
+	2:13 mark(:)
+	3:16 in()
+	3:16 ident(foo)
+	3:20 mark(:)
+	3:22 ident(number)
+	3:29 mark(=)
+	3:31 number(45)  45
+	4:16 newline()
+	4:16 ident(bar)
+	4:20 mark(:=)
+	4:23 string("string")  string
+	5:8 newline()
+	5:8 out()
+	5:8 newline()
+	5:8 ident(program)
+	5:15 mark(:)
+	6:16 in()
+	6:16 ident(foo)
+	6:20 mark(:=)
+	6:23 number(4)  4
+	7:16 newline()
+	7:16 ident(print)
+	7:22 ident(foo)
+	7:25 mark(,)
+	7:27 ident(bar)
+	8:0 newline()
+	8:0 out()
+	8:0 newline()
+	8:0 out()
+	8:0 newline()
+	8:0 eof()
diff --git a/csrc/scanner.mdc b/csrc/scanner.mdc
index 21ec828..80adb2a 100644
--- a/csrc/scanner.mdc
+++ b/csrc/scanner.mdc
@@ -854,7 +854,8 @@ information and return one token.
 		    state->indent_level < sizeof(state->indent_sizes)-1) {
 			state->indent_level += 1;
 			state->indent_sizes[state->indent_level] = state_indent(state);
-			state->delayed_lines -= 1;
+			if (state->delayed_lines)
+				state->delayed_lines -= 1;
 			tk.num = TK_in;
 			return tk;
 		}