]> ocean-lang.org Git - ocean/log
ocean
10 years agoparsergen: fix a couple of typos in text. linebreakparser
NeilBrown [Sun, 4 May 2014 10:41:50 +0000 (20:41 +1000)]
parsergen: fix a couple of typos in text.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: improve tracing of parse for line-oriented details.
NeilBrown [Sun, 4 May 2014 10:40:35 +0000 (20:40 +1000)]
parsergen: improve tracing of parse for line-oriented details.

Report if each state is known to start a line, and if each
frame permits a newline.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: track when newline is permitted, and discard if not.
NeilBrown [Sun, 4 May 2014 10:31:05 +0000 (20:31 +1000)]
parsergen: track when newline is permitted, and discard if not.

A newline is only permitted (as a recognised symbol) if we are
parsing a non-indented line-like segment.
If we have seen an internal indent since the last line-like start,
newline tokens should be ignored.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: compute starts_line for each state.
NeilBrown [Sun, 4 May 2014 10:21:00 +0000 (20:21 +1000)]
parsergen: compute starts_line for each state.

Using the per-symbol "can_eol" we can deduce for each state whether
it is expected to (sometime) start a line-oriented syntax element.

The flag is "starts_line" and is made available to the parser.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: compute "can_eol" for each symbol.
NeilBrown [Sat, 3 May 2014 21:55:03 +0000 (07:55 +1000)]
parsergen: compute "can_eol" for each symbol.

A symbol is "can_eol" if it can derive a phrase which ends with a
newlike token.
This will allow us to recognise line-like sections of code and
thus know when to ignore newlines and when not to.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: add handling for TK_IN and TK_OUT indentparser
NeilBrown [Sun, 24 Nov 2013 06:54:02 +0000 (17:54 +1100)]
parsergen: add handling for TK_IN and TK_OUT

Intents are tracked.  The end of an indented region forces certain
reductions.  And indents are managed during error handling.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: centralise (some of) the collecting of next token.
NeilBrown [Sun, 24 Nov 2013 07:14:11 +0000 (18:14 +1100)]
parsergen: centralise (some of) the collecting of next token.

A future patch will introduce next sites where we want to
discard the current token.
Rather than calling "token_next" at each site, make it possible
to just set "tk = NULL", and the next token will automatically
be collected when needed.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: recorded a prefered shift-symbol for error recovery.
NeilBrown [Sun, 24 Nov 2013 06:50:09 +0000 (17:50 +1100)]
parsergen: recorded a prefered shift-symbol for error recovery.

When we find there is no valid parse step, one option that we don't
currently try is to synthesize a symbol and shift it.  i.e. insert a
missing symbol.

A future patch will provide a circumstance where this is the ideal
response, so here we choose a symbol which could usefully be shifted.
We choose the one that will get us closest to the end of a production.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsegen: unify the "next" frame to go onto stack.
NeilBrown [Sun, 24 Nov 2013 06:36:41 +0000 (17:36 +1100)]
parsegen: unify the "next" frame to go onto stack.

We current have a current 'state' in the parser and a 'sym'
which is a local variable passed around in different ways.
Both of these get pushed onto the stack at the next 'shift'.

We will shortly add some more fields to the stack frame, so unify
'state' and 'sym' in to a 'next' struct in the parser struct which can
easily be extended.

Signed-off-by: NeilBrown <neilb@suse.de>
10 years agoparsergen: various cosmetic fixes
NeilBrown [Sun, 24 Nov 2013 05:53:34 +0000 (16:53 +1100)]
parsergen: various cosmetic fixes

- various typos
- remove double spaces
- re-arrange some text slightly.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: report some tokens better when tracing.
NeilBrown [Thu, 25 Jul 2013 10:23:26 +0000 (20:23 +1000)]
parsergen: report some tokens better when tracing.

Some tokens are best traced by giving their name rather
than their content.  e.g. newline.  So make a special
case of those.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoRefresh boot-strap files.
NeilBrown [Thu, 25 Jul 2013 10:15:43 +0000 (20:15 +1000)]
Refresh boot-strap files.

There have been some changes to mdcode.mdc, so time to
update the generated files in 'boot-strap'

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoRename Indent and Undent to IN and OUT
NeilBrown [Thu, 25 Jul 2013 10:13:22 +0000 (20:13 +1000)]
Rename Indent and Undent to IN and OUT

These names work better for me.
There is an indent on every line, so the place where the
indent increases shouldn't be called "indent".
And "undent" isn't a word.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: remove 'depth' arg from do_reduce.
NeilBrown [Sun, 21 Jul 2013 10:44:07 +0000 (20:44 +1000)]
parsergen: remove 'depth' arg from do_reduce.

This was only used for tracing and now tracing is done differently.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: allow "$void" to remove current type. draftparser
NeilBrown [Sun, 21 Jul 2013 09:27:27 +0000 (19:27 +1000)]
parsergen: allow "$void" to remove current type.

This allows return to the initial state where nonterminals
have no type.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: improve tracing.
NeilBrown [Sun, 21 Jul 2013 09:18:09 +0000 (19:18 +1000)]
parsergen: improve tracing.

Change the tracing to print the full stack after every step.
This can be cross-referenced with the report that parsergen
can generate to get a full picture of how the parse is progressing.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: make sure we continue making states until all done.
NeilBrown [Sun, 21 Jul 2013 08:20:00 +0000 (18:20 +1000)]
parsergen: make sure we continue making states until all done.

Whenever we add a state, we need to check again, as it might have been
added early in the list.
There is probably a much more efficient way to do this....

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: change sort order for items.
NeilBrown [Sun, 21 Jul 2013 08:10:45 +0000 (18:10 +1000)]
parsergen: change sort order for items.

I want items with larger indexes to always preceed smaller
indexes.  This makes the report easier to follow.
So negate the index number (and add an offset) when sorting.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: change symset function to use 'unsigned short'.
NeilBrown [Sun, 21 Jul 2013 08:04:24 +0000 (18:04 +1000)]
parsergen: change symset function to use 'unsigned short'.

To make full use of all 16 bits of a sym, we should make sure
we consistent use "unsigned".

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen: fix bug testing return value for shift()
NeilBrown [Sun, 21 Jul 2013 08:00:13 +0000 (18:00 +1000)]
parsergen: fix bug  testing return value for shift()

shift() returns 0 on failure, not negative.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoparsergen - adjust for recent scanner fix.
NeilBrown [Sun, 21 Jul 2013 07:58:34 +0000 (17:58 +1000)]
parsergen - adjust for recent scanner fix.

scanner was recently fixed to correctly skip over C code with
a string immediately after a '(', so we can remove the extra
spaces we inserted before.

For this to work, we need to stop ignoring TK_String

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner: don't allow an unknown mark to run into a string or comment
NeilBrown [Sun, 21 Jul 2013 07:53:51 +0000 (17:53 +1000)]
scanner: don't allow an unknown mark to run into a string or comment

This means that e.g.
   printf("hello")
where no marks are declared will not treat
    ("
as an known mark, but instead find
    (
and then a string.

This is important for skipping over C code in 'parsergen'.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner: initialise token state properly.
NeilBrown [Sun, 21 Jul 2013 07:52:39 +0000 (17:52 +1000)]
scanner: initialise token state properly.

We need to call do_strip() at the start to ensure the correct
col and indent is used for the first token.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner: fix do_strip bug.
NeilBrown [Sun, 21 Jul 2013 07:51:37 +0000 (17:51 +1000)]
scanner: fix do_strip bug.

do_strip wasn't striping tabs properly

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoMakefile: if bootstrap was required, suggest "make" be run again
NeilBrown [Fri, 12 Jul 2013 21:25:00 +0000 (07:25 +1000)]
Makefile: if bootstrap was required, suggest "make" be run again

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoUpdate .gitignore with new build targets
NeilBrown [Fri, 12 Jul 2013 21:23:03 +0000 (07:23 +1000)]
Update .gitignore with new build targets

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoNew file: parsergen
NeilBrown [Fri, 12 Jul 2013 21:21:37 +0000 (07:21 +1000)]
New file: parsergen

This reads and analyses a grammar and generates a parser.

It include a simple calculator

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner: must call close_token before returning the token.
NeilBrown [Fri, 12 Jul 2013 21:18:06 +0000 (07:18 +1000)]
scanner: must call close_token before returning the token.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner/numbers: fix typo that broken positive exponents.
NeilBrown [Fri, 12 Jul 2013 21:16:07 +0000 (07:16 +1000)]
scanner/numbers: fix typo that broken positive exponents.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner: make the array of known words "const"
NeilBrown [Fri, 12 Jul 2013 21:15:32 +0000 (07:15 +1000)]
scanner: make the array of known words "const"

After all, it shouldn't change.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agomdcode: rename code_print to code_node_print and export it.
NeilBrown [Fri, 12 Jul 2013 21:13:55 +0000 (07:13 +1000)]
mdcode: rename code_print to code_node_print and export it.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoFix typo in mdcode.mdc
NeilBrown [Fri, 12 Jul 2013 21:12:21 +0000 (07:12 +1000)]
Fix typo in mdcode.mdc

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoscanner.mdc: lexical scanner for Ocean.
NeilBrown [Sat, 22 Jun 2013 09:18:55 +0000 (19:18 +1000)]
scanner.mdc: lexical scanner for Ocean.

This scanner does lexical analysis and produces tokens.
It also handles numbers and escapes in strings.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoMakefile: auto-support .mdc files.
NeilBrown [Sat, 22 Jun 2013 09:17:49 +0000 (19:17 +1000)]
Makefile: auto-support .mdc files.

Any *.mdc cause "md2c" to be run to create a "*.mk" file which is
included and used.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agoRecord in a code_node whether it was indented or not.
NeilBrown [Sat, 22 Jun 2013 07:39:23 +0000 (17:39 +1000)]
Record in a code_node whether it was indented or not.

This is needed to correctly adjust for this indent when processing the
code.

Signed-off-by: NeilBrown <neilb@suse.de>
11 years agomdcode, md2c - extract C code from a literate markdown program
NeilBrown [Wed, 5 Jun 2013 20:20:35 +0000 (06:20 +1000)]
mdcode, md2c - extract C code from a literate markdown program

All C programs here will be written in literate style using markdown.
md2c strips out the C code so that it can be compiled.

Signed-off-by: NeilBrown <neilb@suse.de>