-### Setting `can_eol`
-
-In order to be able to ignore newline tokens when not relevant, but
-still include them in the parse when needed, we will need to know
-which states can start a "line-like" section of code. We ignore
-newlines when there is an indent since the most recent start of a
-line-like section.
-
-To know what is line-like, we first need to know which symbols can end
-a line-like section, which is precisely those which can end with a
-newline token. These symbols don't necessarily alway end with a
-newline, but they can. Hence they are not described as "lines" but
-only "line-like".
-
-Clearly the `TK_newline` token can end with a newline. Any symbol
-which is the head of a production that contains a line-ending symbol
-followed only by nullable symbols is also a line-ending symbol. We
-use a new field `can_eol` to record this attribute of symbols, and
-compute it in a repetitive manner similar to `set_nullable`.
-
-###### symbol fields
- int can_eol;
-
-###### functions
- static void set_can_eol(struct grammar *g)
- {
- int check_again = 1;
- g->symtab[TK_newline]->can_eol = 1;
- while (check_again) {
- int p;
- check_again = 0;
- for (p = 0; p < g->production_count; p++) {
- struct production *pr = g->productions[p];
- int s;
-
- if (pr->head->can_eol)
- continue;
-
- for (s = pr->body_size - 1; s >= 0; s--) {
- if (pr->body[s]->can_eol) {
- pr->head->can_eol = 1;
- check_again = 1;
- break;
- }
- if (!pr->body[s]->nullable)
- break;
- }
- }
- }
- }
-