From: NeilBrown <neil@brown.name>
Date: Wed, 10 Mar 2021 00:00:23 +0000 (+1100)
Subject: parsergen: split out the "shift or ignore" section of parsing.
X-Git-Url: https://ocean-lang.org/code/?p=ocean;a=commitdiff_plain;h=4dcb5e6f58a1d1b830787f1b6cec7ac5f3cc1668

parsergen: split out the "shift or ignore" section of parsing.

This will make it easier to document.

Signed-off-by: NeilBrown <neil@brown.name>
---

diff --git a/csrc/parsergen.mdc b/csrc/parsergen.mdc
index 6eb4318..4708e0c 100644
--- a/csrc/parsergen.mdc
+++ b/csrc/parsergen.mdc
@@ -2768,6 +2768,43 @@ in the thing that preceed:
 Here the NEWLINE will be shifted because nothing can be reduced until
 the `if` is seen.
 
+For other tokens, we shift the next token if that is possible, otherwise
+we try to reduce a production.
+
+###### try shift or ignore
+
+	if (tk->num == TK_in) {
+		free(tk);
+		tk = NULL;
+		parser_trace_action(trace, "Record");
+		continue;
+	}
+	if (tk->num == TK_out) {
+		if (1) {
+			// OK to cancel
+				free(tk);
+			tk = NULL;
+			parser_trace_action(trace, "Cancel");
+			continue;
+		}
+		// fall through to error handling as both SHIFT and REDUCE
+		// will fail.
+	}
+	if (tk->num == TK_newline) {
+		if (1) {
+			free(tk);
+			tk = NULL;
+			parser_trace_action(trace, "Discard");
+			continue;
+		}
+	}
+	if (shift(&p, tk->num, tk, states)) {
+		tk = NULL;
+		parser_trace_action(trace, "Shift");
+		## did shift
+		continue;
+	}
+
 We have already discussed the bulk of the handling of a "reduce" action,
 with the `pop()` and `shift()` functions doing much of the work.  There
 is a little more complexity needed to manage storage for the asn (Abstract
@@ -2875,39 +2912,7 @@ dropping tokens until either we manage to shift one, or reach end-of-file.
 			parser_trace(trace, &p,
 			             tk, states, non_term, config->known_count);
 
-			if (tk->num == TK_in) {
-				free(tk);
-				tk = NULL;
-				parser_trace_action(trace, "Record");
-				continue;
-			}
-			if (tk->num == TK_out) {
-				if (1) {
-					// OK to cancel
-
-					free(tk);
-					tk = NULL;
-					parser_trace_action(trace, "Cancel");
-					continue;
-				}
-				// fall through to error handling as both SHIFT and REDUCE
-				// will fail.
-			}
-			if (tk->num == TK_newline) {
-				if (1) {
-					free(tk);
-					tk = NULL;
-					parser_trace_action(trace, "Discard");
-					continue;
-				}
-			}
-			if (shift(&p, tk->num, tk, states)) {
-				tk = NULL;
-				parser_trace_action(trace, "Shift");
-				## did shift
-				continue;
-			}
-
+			## try shift or ignore
 			## try reduce
 			## handle error
 		}