NeilBrown [Sun, 9 Jun 2019 22:35:36 +0000 (08:35 +1000)]
parsergen: include virtual symbols in table of non-terminals
Symbol numbers assigned in grammar_analyse are in three groups:
- predefined (NUMBER, STRING, etc)
- Terminals
- everything else: non-terminals and virtual.
When creating the non_term[] list of names, we need to include
virtual symbols in there, otherwise lookup by symbol-number
might find the wrong value - or might reach beyond end of array.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 9 Jun 2019 05:55:21 +0000 (15:55 +1000)]
oceani: labels only in 'use' statement.
A variable is only set as a label if it appear at top
level of "use" statement - otherwise it is "unknown".
This makes error messages more meaningful.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 9 Jun 2019 00:15:39 +0000 (10:15 +1000)]
oceani: add more syntax error handling.
We catch some errors earlier so more errors can be
reported in a single run.
The indent structure make recovery easy!
As we might abort out of a var scope, we only insist the scope
is balanced in there are no parser errors.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 9 Jun 2019 00:03:44 +0000 (10:03 +1000)]
scanner: hide impossible code from coverage testing.
The code in "white space" section is truly impossible, as
we don't return WEOF untile ->node is NULL, and when that
happens, ->col is set to 0.
Other code is only "mostly dead".
This brings coverage to 95%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 23:54:39 +0000 (09:54 +1000)]
scanner: test for errors with multi-line objects crossing node.
multi-line strings and comments musn't cross nodes.
Testing this increases coverage above 94%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 23:42:05 +0000 (09:42 +1000)]
scanner: improve transition from node to node.
When we are at the end of a node, it is wrong to use do_strip() as
that looks beyond the end of the node.
It is better, once we have determined to accept the newline at the
end of a node (i.e. once no unget is possible), to move to the
start of the next node, and assess column position and indents from
that perspective.
Do this removes some tests on at_son/at_eon, and make some code a
bit more transparent - for example the flag that say whether an "out"
is next now depends on where a newline was recently seen, which makes
more sense than whether we were at the start of a node (out and newline
alternate in some contexts).
Also: add the test which found this problem. This requires a
new set of tests - tests which can scan tokens from multiple nodes.
Now that we are testing node transitions, the coverage has jumped
over 92%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 10:27:15 +0000 (20:27 +1000)]
scanner: allow a section to be specified.
If --section arg is given, only report on that section.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 10:18:35 +0000 (20:18 +1000)]
scanner: fix bug with indent at start of node.
If we find an indent, we assume there are delayed newlines
to comsume.
This is often true, but not at the start of a node.
So don't decrement delayed_lines if it is already zero.
Add the test case that found this bug for me.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 04:35:34 +0000 (14:35 +1000)]
scanner: fix handling of indents in sub-nodes
I seem to have confused ->indent_sizes[] and ->col
->col is used for the reported location of a token so
must be the actual column in the file, with no adjustment.
->indent_sizes[] is indents, which must include any inherited from
parent nodes. So this is a completely different value.
So change mdcode to store the local node indent in ->needs_strip -
this is the number of text columns that are stripped off.
This, subtracted from ->indent is the text offset of the physical
start-of-line. Adding the measured ->col then gives us
the indent in the composed file, the indent that must be used
for detecting TK_in and TK_out.
Introduce a new function state_indent() which determines that indent,
and use it instead of ->col.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 04:26:15 +0000 (14:26 +1000)]
scanner: fix at_son()
Current test for "at start of node" is broken for 2 reasons.
1/ it doesn't account for the node-indent chars that are stripped
off by do_strip()
2/ it check the ->offset *after* a character has been extracted,
it needs to check the offset from before, which is in ->prev_offset
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 8 Jun 2019 04:23:05 +0000 (14:23 +1000)]
scanner: fix some typos in text
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Thu, 6 Jun 2019 22:51:23 +0000 (08:51 +1000)]
parsergen: allow prefix code for do_reduce to be provided.
This allow common variables to be declared and initialised
for all reduce fragments.
Use this in oceani to declare 'c' holding the parser context,
so we don't need to call config2context in so many places.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Thu, 6 Jun 2019 22:31:07 +0000 (08:31 +1000)]
oceani: setting ->parse_error now aborts type analysis.
When ever we call type_err() or tok_err(), in type analysis,
we currently need to also set *ok=0; This is uhly and error-prone.
So instead, we can detect ->parse_error being set, and assume
something is no OK.
This is most easily done in a wrapper for propagate_types.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 5 Jun 2019 08:21:18 +0000 (18:21 +1000)]
oceani: redo parsing of blank lines.
I've been puzzling how best to write a grammar to
handle blank lines and option line-breaks well.
The "OptNL" approach didn't work, and "Newlines"
only sometimes works.
I won't try to explain all the logic here, but I do plan to write a
blog post about it soon.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 5 Jun 2019 08:12:20 +0000 (18:12 +1000)]
oceani-tests: use LALR to check for conflicts.
LR1 analysis takes a lot longer than LALR and will never find
more conflicts, only ever fewer.
So with to that when checking for conflicts.
Also report that the check is happening, and fix a typo.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 5 Jun 2019 08:09:59 +0000 (18:09 +1000)]
parsergen - don't completely hide non-critical conflicts.
Shift/Reduce conflicts that are likely to be resolved by
a line break are currently hidden. This is probably a good idea,
but sometimes it can be useful to see them anyway.
So report them as "non-critical" conflicts, and don't
count them - so if no other conflicts are found, the
"no conflicts" message is still generated.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 2 Jun 2019 06:45:56 +0000 (16:45 +1000)]
parsegen: Add brief explanation about optional newlines.
Optional newlines need care with a parsergen parse and the special
rules around them mean you cannot have a symbol that just absorbs
newlines. Rather, you need to put the newline absorbtion in front of
whatever is allowed to follow newlines.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 11:54:53 +0000 (21:54 +1000)]
oceani: allow spaces in numbers.
Spaces are now allowed in (decimal) numbers to highlight
grouping.
Tests are added.
Also ensure unknown marks are reported as errors.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 11:51:22 +0000 (21:51 +1000)]
scanner: improve number parsing.
In particular, space must be preceeded and followed by a digit
(not a letter).
Also '_' must be preceded and followed by a hex digit, but this
wasn't enforced.
Add tests to check on numbers more thoroughly.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 08:31:55 +0000 (18:31 +1000)]
Remove excess blank lines
We never need 2 blank lines in a row.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 08:27:47 +0000 (18:27 +1000)]
scanner tests: handle errors and more
Handle assorted more cases, particularly errors.
Now coverage has passed 90%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 08:25:50 +0000 (18:25 +1000)]
scanner: handle completely unrecognized characters.
If we are ignoring numbers, then a digit looks like
nothing at all, not even an unknown mark.
Make sure to handle that properly.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 08:19:54 +0000 (18:19 +1000)]
scanner: handle unknown marks once.
There were two places in the code that tried to handle
unknown marks. We only need one.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 29 May 2019 00:38:39 +0000 (10:38 +1000)]
Separate demos from tests.
'tests' check that the code is working, and fail if the results
aren't what was expected.
'demos' simply run the code and show what it can do. The don't provide
any immediate assurance that it is doing the right thing.
Separate these into "make tests" and "make demos"
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 28 May 2019 05:57:09 +0000 (15:57 +1000)]
Add test code for the scanner
Currently only 85% coverage. I haven't tested error yet,
or various other things. I have found bugs though!
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 28 May 2019 05:49:36 +0000 (15:49 +1000)]
scanner: fix typo in testing ignored flagged
Use '&' to test a bit, not '&&' !!!
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 28 May 2019 05:04:35 +0000 (15:04 +1000)]
scanner: fix multi-line strings
I broke multi-line strings. A quote cannot be part of an
unknown mark, but it can be part of '"""' or "'''".
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 26 May 2019 22:48:35 +0000 (08:48 +1000)]
scanner: add options to allow more complete testing.
Allow all facets of scanner to be controlled by command line
options. This will make it easier to do exhaustive testing.
Also make sure to free things that are allocated.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 26 May 2019 05:04:43 +0000 (15:04 +1000)]
parsergen - fix newline parsing (again)
Add a test-case to oceani-tests.mdc which fails, but shouldn't.
It fails because expressions are treated as line-like, so newlines
aren't ignored.
I realize that having linelike symbols being those that are followed by
a newline really doesn't work.
So go back to the original idea that "linelike symbols are those which
contain a newline".
Then a state starts a line if it is at the start of a linelike symbol.
This simplifies the code, seems to work correctly for existing tests,
and allows the new test to pass.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 26 May 2019 05:03:13 +0000 (15:03 +1000)]
parsergen: make it easier to test the simple 'calc' code.
Allow 'calc' to read a test code from parsergen.mdc.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 20:46:13 +0000 (06:46 +1000)]
Oceani - Jamison Creek Version
Clean up text and provide new version name.
Additions for this version include:
- type identifiers
- arrays and structs
- global const
- "and then" "or else" "if .. else"
- test suite
- valgrind testing, coverage testing
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 15:39:23 +0000 (01:39 +1000)]
oceani-tests: add tests for str/bool command line args
This brings test coverage over 94%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 15:06:34 +0000 (01:06 +1000)]
oceani: cleanup "sayhello" output
This had become unreadable due to the long arrays.
A few other little cleanups and comments.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 14:16:01 +0000 (00:16 +1000)]
oceani: add structs
This is just basic struct support, showing how they
are declared and used.
There is more sophistication to be added such as anonymous
fields and per-field attributes.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 13:51:24 +0000 (23:51 +1000)]
oceani: fix merging of conditionally-scoped variables.
The problem here was that the list seen in ->in_scope
includes more than just what is currently in-scope.
It also contains things that have been replaced by new instances
of the name.
These can be detected a they aren't the first variable listed
under their name any more.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 02:21:55 +0000 (12:21 +1000)]
oceani-tests: delay valgrind tests
As valgrind can be slow, separate the valgrind tests out to
after the main tests. That was failing that can be found quickly
are found quickly.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 00:49:19 +0000 (10:49 +1000)]
oceani: fix indent for Xval propagate
this is double-indented. I wonder why.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 00:11:43 +0000 (10:11 +1000)]
oceani-tests: test code that has been printed
Test that the printed code actually works, as well as being re-printable.
Also simplify the messages so they don't use as much space
and fix a typo "exit1" -> "exit 1"
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 18 May 2019 00:01:30 +0000 (10:01 +1000)]
oceani: print pre-declared constants when printing program.
The program is no more than just an 'exec'. We need to print
the declarations too. As yet, there are only constants.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Fri, 17 May 2019 13:31:48 +0000 (23:31 +1000)]
scanner: handle missing newline at EOF
If there is no newline at EOF, we can see EOF immediately after
a valid symbol. This can lead to calling close_token() when
state->node is NULL, which crashes.
The code in close_token() only makes sense if state->node is still the
same as token->node. If it isn't, the token must be at the very end of
its code-node, so a different calculation is needed.
This avoids the NULL deref.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 13 May 2019 06:17:12 +0000 (16:17 +1000)]
oceani: add the option for "const" sections
These are introduced with "const" and can define one or
more constants - on one line with ';' or multiple lines
with {} or :
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 12 May 2019 05:03:27 +0000 (15:03 +1000)]
oceani: allow list of declarations as top level structure
We can have constants, types, etc at top level, as well
a program
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 11 May 2019 01:41:59 +0000 (11:41 +1000)]
oceani: mark code that doesn't need testing.
Some code is included to check and report impossible
conditions. Failure to exercise this code shouldn't
be seen as a failure of test coverage.
So mark such code as //NOTEST and exclude it
from statistics.
This brings test coverage up to 93%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 11 May 2019 01:30:30 +0000 (11:30 +1000)]
oceani: move "complex types" earlier.
I think this fits better before "language elements",
as part of "data structures". Nearly everything it needs
is defined by there.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Fri, 10 May 2019 10:25:01 +0000 (20:25 +1000)]
oceani: fix valgrind-reported errors.
The test suite wasn't running valgrind in a useful way, and wasn't
actually reporting errors.
So fix it to report leaks and unfreed memory, and then fix the
bugs that it finds.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Thu, 9 May 2019 12:05:23 +0000 (22:05 +1000)]
oceani tests: check for conflicts.
If the grammar, when analysed as LR1, contains conflicts, then
don't proceed until they are resolved.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Thu, 9 May 2019 11:59:07 +0000 (21:59 +1000)]
parsergen: don't report expected shift/reduce conflicts.
If a NEWLINE is being treated as a grammar symbol which will sometimes
force a reduction, then an apparent SHIFT/REDUCE conflict that could be
resolved to REDUCE by a newline and would default to shift, is expected
and not really a conflict. So don't report those.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Thu, 9 May 2019 11:13:16 +0000 (21:13 +1000)]
oceani: add conditional expression
[value] if [condition] else [alternate value]
is a conditional expression which will only evaluate and return one of
the two values, depending on the condition.
This has lowest precedence of all expressions.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 8 May 2019 10:50:30 +0000 (20:50 +1000)]
oceani: add "and then" and "or else"
"and then" and "or else" are short-circuit versions
of "and" and "or" - they only evaluate second arg if it
is needed.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 8 May 2019 07:22:53 +0000 (17:22 +1000)]
oceani-tests: assorted more tests.
Assorted tests chosen after examining lines of
code that were not being run.
This pushes the coverage over 90% !!!
Also a bugfix - test suites are good !!!
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 6 May 2019 08:54:40 +0000 (18:54 +1000)]
oceani-tests: add tests for various type errors.
Now we have test coverage for all tok_err and type_err
calls, and have exceeded 85% total.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 6 May 2019 07:57:05 +0000 (17:57 +1000)]
oceani-tests: add tests for various token-errors
This pushes our coverage over 80%
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 6 May 2019 07:38:17 +0000 (17:38 +1000)]
oceani-tests: add tests for error cases.
Only one test so far, but this infrastructure will be useful
for others.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 5 May 2019 03:25:39 +0000 (13:25 +1000)]
oceani-tests: add tests for bad command line args.
Ensuring the correct behaviour with bad command line args
helps increase coverage.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 5 May 2019 03:18:46 +0000 (13:18 +1000)]
mdcode: indent must start with a TAB
For Makefiles, indent must start with a TAB. So
insert a TAB if there are 8 or more spaces of indent.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 5 May 2019 01:56:59 +0000 (11:56 +1000)]
oceani-tests: add test suite.
This is the beginning of a test suite. It guards
against errors, memory leaks, etc.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 5 May 2019 01:51:08 +0000 (11:51 +1000)]
mdcode: don't include blank lines at end of section.
completely blank lines (no indent) at the end of a section
should not be considered part of that section, but rather
separation between this and the next section.
So don't include them in the code.
When a section is used, for example, as sample output to
test against actual output in a test suit, having the
stray blank at the end can be problematic.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 5 May 2019 01:38:22 +0000 (11:38 +1000)]
mdcode: allow a specific section to be extracted.
Previously, the md2c tool only extracted "File:" sections.
Sometimes we need more control, so allow a second argument
to identify a single section to be extracted. This will
be written to stdout.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 1 May 2019 09:01:48 +0000 (19:01 +1000)]
oceani: add array type.
I can now declare arrays and access or assign
members there-of.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 1 May 2019 08:22:38 +0000 (18:22 +1000)]
oceani: prepare for adding new types with new syntax.
1/ 'Variable' now returns an 'exec', so other things returning
and 'exec' can be the LHS of an assignment.
2/ provide forward-decls of generic functions so that
new type functions can use them to work with
components of the types.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 1 May 2019 08:13:21 +0000 (18:13 +1000)]
oceani: add "remainder" operator.
a % b
treats a and b as integers, does a division, and reports
the remainder.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 30 Apr 2019 06:24:09 +0000 (16:24 +1000)]
oceani: add some support for lvalues.
New linterp_exec() returns an lvalue, (pointer to value)
and individual cases can provide either an lvalue or an rvalue.
This means the RHS of assignment might not be a simple variable,
so we leave the "v = v->merged" part to the "interp exec" for Xvar.
Also, linterp_exec() might return NULL, so be careful.
This might happen, e.g., when an array index is out of bounds.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 30 Apr 2019 06:23:21 +0000 (16:23 +1000)]
oceani: fix up test program.
The messages being printed didn't make sense - weird.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 30 Apr 2019 04:00:03 +0000 (14:00 +1000)]
oceani: delay constant assignment test to type analysis.
When we get more complex types, it will be easier to
guard against assigning to a constant during type analysis.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 04:08:23 +0000 (14:08 +1000)]
oceani: allow a type to control how it is printed.
this is needed for e.g. printing the program when there
are declared types.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 03:45:26 +0000 (13:45 +1000)]
oceani: delegate type compatability to type.
Rename vtype_compat to type_compat, and have it
call the ->compat function if the required type
provides it.
Otherwise require type equivalence.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 03:10:15 +0000 (13:10 +1000)]
oceani: introduce type->prepare()
During parsing and analysis we don't want to "init" a
variable as that can allocate memory - when we add arrays,
we might not know yet how much memory.
So introduce 'prepare' to prepare a value - such that calling
free on it will work - without allocating.
'init' is then called when a variable is declared - unless something
is assigned to it instead.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 01:10:03 +0000 (11:10 +1000)]
oceani: disallow assignment if no 'dup' operation.
'dup' is needed for assignment, so if it isn't present
it must be forbidden.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 00:40:12 +0000 (10:40 +1000)]
oceani: allow a variable to be declared with no value
var:type
is now allowed. "var:" is not - there must be either a
type or an initial value.
If only the type is given, the type's 'init' function
is called to provide an initial value.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 28 Apr 2019 00:24:38 +0000 (10:24 +1000)]
oceani: := is no longer a token.
It was always intended that a type could come between
the : and = of a declaration.
It makes more sense, and simplifies the grammar, if
we stop treating := as ever being a token.
So now : and = are separate tokens.
If the parse sees ":=", it will happily treat that
as two separate tokens, which is what we want.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 27 Apr 2019 23:21:06 +0000 (09:21 +1000)]
oceani: make "Base Types" a subsection of "Types"
I don't think it deserved to stand alone.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 27 Apr 2019 23:19:44 +0000 (09:19 +1000)]
oceani: move free_value() to the other generic value code.
This doesn't belong wit the base types.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 22 Apr 2019 12:01:06 +0000 (22:01 +1000)]
oceani - assorted cleanups.
Reading through the whole document found a few typos,
and few things in the wrong place, some areas where
a better explanation would help etc.
This patch combines several such improvements.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 22 Apr 2019 01:17:03 +0000 (11:17 +1000)]
oceani: initial support for named types.
The base types are now stored in a symbol table (linked list)
and can be fetched by name.
name:type = value
now works to declare 'name' as of the given type.
The narrative needs to be improved to include a clear
section on types, and variable declarations of the form
name:type
with no assignment still need to be added.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 20 Apr 2019 06:18:18 +0000 (16:18 +1000)]
oceani: make set of types extensible.
Rather than a enum listing allowed types, we now
have a 'struct type' which can contain any type.
For now it just has an enum and function pointer
to the existing function, but that can be extended.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 20 Apr 2019 04:57:21 +0000 (14:57 +1000)]
oceani: fix a couple of typos.
There is always one more typo...
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 20 Apr 2019 04:40:52 +0000 (14:40 +1000)]
oceani: discard Vnolabel infavour of rules.
Instead Vnolabel as a type, change "bool_permitted" to
a set of rules (Rboolok) and add a new rule: Rnolabel.
This requires changes to type_err() and elsewhere.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 2 Apr 2019 01:36:35 +0000 (12:36 +1100)]
oceani: don't store 'tail' in 'struct value'.
It is never used so there isn't much point
storing it. If/When we do use it, we'll
parse it immediately and store the meaning.
And in any case, it should be char[3].
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Fri, 22 Mar 2019 09:23:59 +0000 (20:23 +1100)]
oceani.mdc: fix assorted typos.
Little errors mostly.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 06:48:44 +0000 (17:48 +1100)]
oceani: detect and report tails on numbers and strings.
Currently illegal, so we should say so.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 06:47:00 +0000 (17:47 +1100)]
scanner: capture the tail of a string.
A string can have a tail,
"hello"xy
where the "xy" might one day have a meaning (utf8? utf16??).
We should capture that.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 06:08:11 +0000 (17:08 +1100)]
oceani: track where each variable was declared.
This allows better error messages when they are
redeclared.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:58:40 +0000 (16:58 +1100)]
oceani: minimal error tracking.
1/ If the parser hits an error, catch it at eof by having
Program -> ERROR
2/ Allow the presence of and error to be recorded in the context.
3/ If an error occured, don't try to run, and do exit with an error
status.
Now, at last, we get told where the error was.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:54:03 +0000 (16:54 +1100)]
oceani: handle NULL from parse_oceani()
If the parsed finds nothing, it will return a pointer
to a NULL. We need to be careful not to deref this
in printing, analysis, or execution.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:46:04 +0000 (16:46 +1100)]
oceani: Expression etc should be 'exec', not 'binode'.
As a 'var' and a 'val' are possible expressions,
'binode' isn't correct. This is obvious when you
consider that I needed to case $1 for a var or var
before assigning it to $0 for Factor -> Value etc.
So change all these to expect the more generic 'struct exec *'.
Without this change, error handling can try to free a var as though it
was a binode, and get into trouble.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:40:06 +0000 (16:40 +1100)]
parsergen: enable error handling.
The error handling code currently aborts early because
it was badly broken.
After recent changes it works well enough for experimenting,
so remove the exit(1) and other unnecessary code, and
let's experiment.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:38:12 +0000 (16:38 +1100)]
parsergen: improve symbol-discard in error handling.
As we don't keep the full look-ahead set, we need to pay a
bit more attention when discarding input symbols, looking
for one we recognize. We need to consider anything
that can be shifted in any state we can reach by simple
shifting.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:31:14 +0000 (16:31 +1100)]
parsergen: be careful shifting TK_error
shift() behaved a little differently when p.tos == 0,
and if the stack is completely empty, there is little
point trying to shift TK_error as there is no state
to work with.
So rearrange the loop slightly.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 05:23:27 +0000 (16:23 +1100)]
parsergen: document min_prefix and starts_line.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 19 Feb 2018 04:32:46 +0000 (15:32 +1100)]
parsergen: remove symbol synthesis option.
This idea never worked, and cannot work as we cannot
magically synthesis the ast node to go with a synthesized
symbol.
If we want to synthesize something on error, we just use
a production like
foo -> ERRROR ${ $0 = a_new_for(); }$
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 18 Feb 2018 05:10:59 +0000 (16:10 +1100)]
scanner: fix calculation of column.
When we stripe the expected indent from the
start of each line, we need to update 'col'
to correctly account for tabs.
Previous code effectively assumed tabs were 4 spaces.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 18 Feb 2018 05:10:12 +0000 (16:10 +1100)]
scanner: improve text and fix typos.
Improve the text about 'marks' to explain the difficulty
with comments and '/'.
Fix a few typos.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 17 Feb 2018 08:20:53 +0000 (19:20 +1100)]
oceani - add error reporting for type errors
This is still rough, but it is at least a
basis to work on.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 17 Feb 2018 08:19:34 +0000 (19:19 +1100)]
scanner: fix silly error calling indent_tab()
Wrong sort of value is passed.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 17 Feb 2018 08:02:57 +0000 (19:02 +1100)]
oceani: pass parse_context through propagate_types
This will allow more context for error messages.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sat, 17 Feb 2018 01:01:08 +0000 (12:01 +1100)]
oceani: fix type analysis for 'while' condition.
The condition in a 'while' can always return Bool,
or may return some other consistent type.
This requires extra subtlety in analysis.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Wed, 31 Jan 2018 03:25:16 +0000 (14:25 +1100)]
New lang: Stoney Creek
This is the second iteration of language design.
it adds scopes variables.
Variables must be declared before use, but they
can be declared in both branches of an 'if', then
used afterwards as the one variable.
No hole-in-scope is allowed: names that are declared
cannot be redeclared in a subordinate scope.
A test program is included:
make sayhello
Note that there are no useful error messages yet.
That is the next step.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Sun, 11 Feb 2018 08:07:55 +0000 (19:07 +1100)]
scanner: fix parsing of comments.
If '/' is a known mark, then "//" and "/*" comments don't get
parsed properly. Add handling for this case.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Mon, 29 Jan 2018 05:20:01 +0000 (16:20 +1100)]
parsergen.mdc: add precedence handling
This hasn't been documented properly in the text yet, but
the example has been changed to work and it seems good.
There is no support for precedence to select between two
reductions because I don't believe that ever happens :-) and
I haven't done anything special for non-associative because
I don't know what I would do.
Signed-off-by: NeilBrown <neil@brown.name>
NeilBrown [Tue, 6 Feb 2018 05:42:01 +0000 (16:42 +1100)]
parsergen: record line number of reduce fragments.
If there is an error in a code fragment used to handle
a 'reduce' action, we need the compiler to report the
correct line from the grammar file.
This information is easily available from the scanner,
we just need to pass it along.
Signed-off-by: NeilBrown <neil@brown.name>