scanner: capture the tail of a string.

A string can have a tail,
   "hello"xy

where the "xy" might one day have a meaning (utf8? utf16??).

We should capture that.

Signed-off-by: NeilBrown <neil@brown.name>

diff --git a/csrc/scanner.mdc b/csrc/scanner.mdc
index a5eeb1f128b5163b1715b263a033a3a64265c73a..0d7a62d05127697cfdfed7ff207dd923de60c69c 100644 (file)
--- a/csrc/scanner.mdc
+++ b/csrc/scanner.mdc
@@ -520,6 +520,10 @@ If `TK_string` is ignored, then quote characters will appear as `TK_mark`s.
                                break;
                        }
                }
+               while (!at_eon(state) && (ch = get_char(state)) &&
+                                         iswalpha(ch))
+                       ;
+               unget_char(state);
                close_token(state, &tk);
                return tk;
        }