Signed-off-by: NeilBrown <neil@brown.name>
When a Newline leads to the next block of code there is a question of
whether the various Newline and OUT/IN tokens should appear to
When a Newline leads to the next block of code there is a question of
whether the various Newline and OUT/IN tokens should appear to
-pbelong to the earlier or later block. This is addressed by processing
+belong to the earlier or later block. This is addressed by processing
the tokens in two stages based on the relative indent levels of the
two blocks (each block has a base indent to which the actual indents
are added).
the tokens in two stages based on the relative indent levels of the
two blocks (each block has a base indent to which the actual indents
are added).
int delayed_lines;
int out_next;
int delayed_lines;
int out_next;
-Generating these tokens involve two separate pieces of code.
+Generating these tokens involves two separate pieces of code.
Firstly we need to recognise white space and count the indents and
newlines. These are recorded in the above state fields.
Firstly we need to recognise white space and count the indents and
newlines. These are recorded in the above state fields.
-Separately we need, on each call to `token_next`, we need to check if
+Separately we need, on each call to `token_next`, to check if
there are some delayed tokens and if so we need to advance the state
information and return one token.
there are some delayed tokens and if so we need to advance the state
information and return one token.
-Tokens make not cross into the next `code_node`, and some tokens can
+Tokens may not cross into the next `code_node`, and some tokens can
include the newline at the and of a `code_node`, we must be able to
easily check if we have reached the end. Equally we need to know if
we are at the start of a node, as white space is treated a little
include the newline at the and of a `code_node`, we must be able to
easily check if we have reached the end. Equally we need to know if
we are at the start of a node, as white space is treated a little