From fa35c2e89220fb586700f934d9beba2f3e806f0f Mon Sep 17 00:00:00 2001 From: NeilBrown Date: Sun, 2 Jun 2019 16:45:56 +1000 Subject: [PATCH] parsegen: Add brief explanation about optional newlines. Optional newlines need care with a parsergen parse and the special rules around them mean you cannot have a symbol that just absorbs newlines. Rather, you need to put the newline absorbtion in front of whatever is allowed to follow newlines. Signed-off-by: NeilBrown --- csrc/parsergen.mdc | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/csrc/parsergen.mdc b/csrc/parsergen.mdc index 94c29df..a733035 100644 --- a/csrc/parsergen.mdc +++ b/csrc/parsergen.mdc @@ -2692,9 +2692,29 @@ is still a work-in-progress. `TK_newline` tokens are ignored unless the top stack frame records that they are permitted. In that case they will not be considered for shifting if it is possible to reduce some symbols that are all since -the most recent start of line. This is how a newline forcible +the most recent start of line. This is how a newline forcibly terminates any line-like structure - we try to reduce down to at most one symbol for each line where newlines are allowed. +A consequence of this is that a rule like + +###### Example: newlines - broken + + Newlines -> + | NEWLINE Newlines + IfStatement -> Newlines if .... + +cannot work, as the NEWLINE will never be shifted as the empty string +will be reduced first. Optional sets of newlines need to be include +in the thing that preceed: + +###### Example: newlines - works + + If -> if + | NEWLINE If + IfStatement -> If .... + +Here the NEWLINE will be shifted because nothing can be reduced until +the `if` is seen. When, during error handling, we discard token read in, we want to keep discarding until we see one that is recognised. If we had a full set -- 2.43.0