Flycheck and syntax tables: erroneous behavior I'm unsure how to fix

Question

In the course of writing a Flycheck checker for ten-hundred-mode's new (and as yet inchoate) permissive mode, I noticed that the checker's overlay behavior was influenced by the currently active syntax table in a way that didn't work out well for my purposes.

Consider the following text:

Now they've done it.

"They've" is not one of the thousand most commonly used words, and my checker will report it as an error at line 1, column 5.

In text-mode, or a major mode which uses a syntax table derived from text-mode-syntax-table, Flycheck will overlay as follows:

Now they've done it.

This is, of course, the correct behavior.

In a mode whose syntax table doesn't derive from text-mode's, though -- notably, any programming mode -- Flycheck instead does this:

Now they've done it.

This appears to occur because ' is a word-constituent character in text-mode-syntax-table and its derivatives, but not in the syntax tables of programming modes. I'm not certain why this affects overlay positioning, but my surmise is that Flycheck's method of inferring where a given overlay should end makes use of syntax classes. In any case, it's trivial to produce both correct and incorrect behavior by switching the syntax table or major mode.

(Since "they" is a permissible word, it's reasonable to suspect that the problem here lies with the checker's matching behavior, and not with Flycheck's overlaying behavior. Indeed, that was my initial suspicion, and I spent a couple of hours looking for a bug in my own code and not finding it, before being reluctantly forced to the conclusion that the problem lay elsewhere.)

Since my checker code only controls what problems are reported to Flycheck, and not how Flycheck chooses to overlay them, I'm somewhat at a loss for how to proceed here. In private code I could advise flycheck-error-region-for-mode to conditionally wrap it in with-syntax-table, or something like that, but this is a publicly distributed module, so advice is off the table. Unfortunately, I'm not seeing any other way to get the behavior I need.

Can someone with more knowledge of Flycheck's internals suggest a way in which I can produce correct overlay behavior regardless of the currently active syntax table?

Answer 1

You cannot easily fix this issue. It's a consequence of how we highlight errors in Flycheck.

Currently Flycheck can only receive a single position from a syntax checker, as a pair of line and column. A single character highlight is all too easy to overlook, however, so Flycheck tries to be smart and expand the highlighting region reasonably.

The exact behaviour is controlled by flycheck-highlighting-mode whose default symbols—which you see here—expands the highlighting to symbol boundaries around the column position.

Symbol boundaries are defined by the syntax table, but there's only one for the entire buffer. If the syntax table turns ' into punctuation this behaviour spills into comments even though these are typically free text.

Hence the behaviour you see here: The symbol around the column reported by your syntax checker simply ends at '.

At some point we will support error ranges in Flycheck which let syntax checkers explicitly specify the whole region for the highlight, but that's still far off. At this point we'll also provide a more flexible API to add Flycheck highlights to the current buffer which will let you programmatically add errors to the current buffer without the additional indirection of a syntax checker.

But meanwhile you simply have to accept this behaviour. Other than advising as you suggested there's nothing that you can do.