-The other allocation stores all other stack fields of which there are six.
-The `state` is the most important one and guides the parsing process. The
-`sym` is nearly unnecessary. However when we want to free entries from the
-`asn_stack`, it helps to know what type they are so we can call the right
-freeing function. The symbol leads us to the right free function through
+ void *memdup(void *m, int len)
+ {
+ void *ret;
+ if (len == 0)
+ return NULL;
+ ret = malloc(len);
+ memcpy(ret, m, len);
+ return ret;
+ }
+
+ static struct token *tok_copy(struct token tk)
+ {
+ struct token *new = malloc(sizeof(*new));
+ *new = tk;
+ return new;
+ }
+
+### The state stack.
+
+The core data structure for the parser is the stack. This tracks all
+the symbols that have been recognised or partially recognised.
+
+The stack usually won't grow very large - maybe a few tens of entries.
+So we dynamically grow an array as required but never bother to
+shrink it down again.
+
+We keep the stack as two separate allocations. One, `asn_stack` stores
+the "abstract syntax nodes" which are created by each reduction. When
+we call `do_reduce` we need to pass an array of the `asn`s of the body
+of the production, and by keeping a separate `asn` stack, we can just
+pass a pointer into this stack.
+
+The other allocation stores all other stack fields of which there are
+several. The `state` is the most important one and guides the parsing
+process. The `sym` is nearly unnecessary as it is implicit in the
+state. However when we want to free entries from the `asn_stack`, it
+helps to know what type they are so we can call the right freeing
+function. The symbol leads us to the right free function through