In the SMIE implementation of SML, ! is given the "expression prefix" syntax class. This seems to lead to a strange tokenization, such that its notion of s-expression (e.g.,if-then-else blocks) gets messed up.

In particular, given the following SML code:

val x = if !foo then bar else baz
val x' = if!foo then bar else baz

val y = if ?foo then bar else baz
val y' = if?foo then bar else baz

try going to the end of each line, after each baz and doing a backward-sexp (C-M-b) twice. In each line except x'’s, you go to the start of baz and then to the top of the if-then-else. On the x' line, you go to the top of the file with the second backward-sexp. I guess the characterisation of ! as an expression prefix causes it to merge with the if and so the grammar never sees an if token at all.

In my analogous situation, should I:

  • just give up on the use of expression prefixes entirely, or
  • fix the custom lexer to split the analogue of the if!foo up into separate tokens (assuming this is possible)?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.