parsergen: split out the "shift or ignore" section of parsing.

This will make it easier to document.

Signed-off-by: NeilBrown <neil@brown.name>

diff --git a/csrc/parsergen.mdc b/csrc/parsergen.mdc
index 6eb43181006fca10c080b545a6f47b85173636ac..4708e0c01c6ce80f7665ae2f02e1a9f9260acdab 100644 (file)
--- a/csrc/parsergen.mdc
+++ b/csrc/parsergen.mdc
@@ -2768,6 +2768,43 @@ in the thing that preceed:
 Here the NEWLINE will be shifted because nothing can be reduced until
 the `if` is seen.
 
+For other tokens, we shift the next token if that is possible, otherwise
+we try to reduce a production.
+
+###### try shift or ignore
+
+       if (tk->num == TK_in) {
+               free(tk);
+               tk = NULL;
+               parser_trace_action(trace, "Record");
+               continue;
+       }
+       if (tk->num == TK_out) {
+               if (1) {
+                       // OK to cancel
+                               free(tk);
+                       tk = NULL;
+                       parser_trace_action(trace, "Cancel");
+                       continue;
+               }
+               // fall through to error handling as both SHIFT and REDUCE
+               // will fail.
+       }
+       if (tk->num == TK_newline) {
+               if (1) {
+                       free(tk);
+                       tk = NULL;
+                       parser_trace_action(trace, "Discard");
+                       continue;
+               }
+       }
+       if (shift(&p, tk->num, tk, states)) {
+               tk = NULL;
+               parser_trace_action(trace, "Shift");
+               ## did shift
+               continue;
+       }
+
 We have already discussed the bulk of the handling of a "reduce" action,
 with the `pop()` and `shift()` functions doing much of the work.  There
 is a little more complexity needed to manage storage for the asn (Abstract
@@ -2875,39 +2912,7 @@ dropping tokens until either we manage to shift one, or reach end-of-file.
                        parser_trace(trace, &p,
                                     tk, states, non_term, config->known_count);
 
-                       if (tk->num == TK_in) {
-                               free(tk);
-                               tk = NULL;
-                               parser_trace_action(trace, "Record");
-                               continue;
-                       }
-                       if (tk->num == TK_out) {
-                               if (1) {
-                                       // OK to cancel
-
-                                       free(tk);
-                                       tk = NULL;
-                                       parser_trace_action(trace, "Cancel");
-                                       continue;
-                               }
-                               // fall through to error handling as both SHIFT and REDUCE
-                               // will fail.
-                       }
-                       if (tk->num == TK_newline) {
-                               if (1) {
-                                       free(tk);
-                                       tk = NULL;
-                                       parser_trace_action(trace, "Discard");
-                                       continue;
-                               }
-                       }
-                       if (shift(&p, tk->num, tk, states)) {
-                               tk = NULL;
-                               parser_trace_action(trace, "Shift");
-                               ## did shift
-                               continue;
-                       }
-
+                       ## try shift or ignore
                        ## try reduce
                        ## handle error
                }