parsergen: document min_prefix and starts_line.

Signed-off-by: NeilBrown <neil@brown.name>

diff --git a/csrc/parsergen.mdc b/csrc/parsergen.mdc
index 48f569168338f159598d532be78f79cd21835c10..905e8f8d125ca7432338059d84dae693ae3597fc 100644 (file)
--- a/csrc/parsergen.mdc
+++ b/csrc/parsergen.mdc
@@ -1174,12 +1174,29 @@ can just compare the symset and the data values together.
                        a.data[i] - b.data[i];
        }
 
+It will be helpful to know if an itemset has been "completed" or not,
+particularly for LALR where itemsets get merged, at which point they
+need to be consider for completion again.  So  a `completed` flag is needed.
+
+For correct handling of `TK_newline` when parsing, we will need to
+know which states (itemsets) can occur at the start of a line, so we
+will record a `starts_line` flag too.
+
+Finally, for handling `TK_out` we need to know where production in the
+current state started *before* the most recent indent.  A state
+doesn't usually keep details of individual productions, so we need to
+add one extra detail. `min_prefix` is the smallest non-zero number of
+symbols *before* DOT in any production in an itemset.  This will allow
+us to determine if the the most recent indent is sufficiently recent
+to cancel it against a `TK_out`.  If it was seen longer ago than the
+`min_prefix`, and if the current state cannot be reduced, then the
+indented section must have ended in the middle of a syntactic unit, so
+an error must be signaled.
+
 And now we can build the list of itemsets.  The lookup routine returns
 both a success flag and a pointer to where in the list an insert
 should happen, so we don't need to search a second time.
 
-FIXME: document min_prefix
-
 ###### declarations
        struct itemset {
                struct itemset *next;
@@ -2686,8 +2703,8 @@ within the stack.  If we can reduce some symbols that are all since
 the most recent indent, then we do that first.  If the minimum prefix
 of the current state then extends back before the most recent indent,
 that indent can be cancelled.  If the minimum prefix is shorter then
-the indent is premature and we must start error handling, which
-currently doesn't work at all.
+the indent had ended prematurely and we must start error handling, which
+is still a work-in-progress.
 
 `TK_newline` tokens are ignored unless the top stack frame records
 that they are permitted.  In that case they will not be considered for
@@ -2764,8 +2781,8 @@ one symbol for each line where newlines are allowed.
                                        parser_trace_action(trace, "Cancel");
                                        continue;
                                }
-                               // fall through and force a REDUCE (as 'shift'
-                               // will fail).
+                               // fall through to error handling as both SHIFT and REDUCE
+                               // will fail.
                        }
                        if (tk->num == TK_newline) {
                                if (!tos->newline_permitted) {