In the SMIE implementation of SML,
! is given the "expression prefix" syntax class. This seems to lead to a strange tokenization, such that its notion of s-expression (e.g.,
if-then-else blocks) gets messed up.
In particular, given the following SML code:
val x = if !foo then bar else baz val x' = if!foo then bar else baz val y = if ?foo then bar else baz val y' = if?foo then bar else baz
try going to the end of each line, after each
baz and doing a
In each line except
x'’s, you go to the start of
baz and then to the top of the if-then-else. On the
x' line, you go to the top of the file with the second backward-sexp. I guess the characterisation of
! as an expression prefix causes it to merge with the
if and so the grammar never sees an
if token at all.
In my analogous situation, should I:
- just give up on the use of expression prefixes entirely, or
- fix the custom lexer to split the analogue of the
if!fooup into separate tokens (assuming this is possible)?