1

I'm going crazy trying to understand why compilation mode and next-error recognize some errors and not others. It appears as though we're long past the days of simple regexps for recognizing errors but I'm not finding anything that explains what is, or is not used now, much less how to debug it.

Why is this recognized as an error:

/Users/kpixley/projects/src-head/cevo/junos/ui/tests/Makefile.inc:5:0 (41): no match found, expected: ":", [ \t] or [\p{Latin}-_.${}/%0123456789]

While this is not?

/Users/kpixley/projects/src-head/cevo/jdid/jdid-infra/build-files/evo/src/Makefile.inc:14:24 (268): rule include_dir: include
4
  • Could you please provide exact details of the compiler you are using? Assuming other assumptions are correct, the errors which are being matched are only being matched by accident, as this error syntax is not actually supported. Can you also confirm whether the two numbers (e.g. 5:0 or 14:24) are indeed LINE:COLUMN numbers?
    – phils
    Commented Jul 25, 2020 at 2:58
  • 5:0 and 14:24 are indeed line & column numbers. The "compiler" here is one I'm developing.
    – kpixley
    Commented Jul 27, 2020 at 16:15
  • Well in that case the solution seems easy -- add a colon after the column number in the output you're generating, and then it will be a format that Emacs recognises. Also consider making it :LINE.COLUMN: which seems to be a well-known format.
    – phils
    Commented Jul 27, 2020 at 20:30
  • Thank you. That will probably be my fix although I very appreciate the elaborations below as well. Thank you all.
    – kpixley
    Commented Jul 28, 2020 at 19:10

2 Answers 2

1

In the following guess-timated answer (tested with Emacs 26.3), I have changed the fourth line from the bottom; i.e., (regexp "[0-9]?") to (regexp "[0-9]+?"). This permits Emacs to match the number 14 following the filename and the first : (colon). To try out this answer, evaluate the Lisp code underneath section labeled THE CODE, and then paste the working data into a scratch buffer and type: M-x compilation-mode

The following link provides other methods for changing an element of an alist: How to replace an element of an alist? I chose to use the solution provided by Dan to modify the alist at issue in this thread.

WORKING DATA:

/Users/kpixley/projects/src-head/cevo/junos/ui/tests/Makefile.inc:5:0 (41): no match found, expected: ":", [ \t] or [\p{Latin}-_.${}/%0123456789]

/Users/kpixley/projects/src-head/cevo/jdid/jdid-infra/build-files/evo/src/Makefile.inc:14:24 (268): rule include_dir: include

THE CODE:

;;;  Load the library before trying to change `compilation-error-regexp-alist-alist'
(require 'compile)

(setf (nth 1 (assoc 'gnu compilation-error-regexp-alist-alist))
  (rx
    bol
    (? (| (regexp "[[:alpha:]][-[:alnum:].]+: ?")
          (regexp "[ \t]+\\(?:in \\|from\\)")))
    (group-n 1 (: (regexp "[0-9]*[^0-9\n]")
                  (*? (| (regexp "[^\n :]")
                         (regexp " [^-/\n]")
                         (regexp ":[^ \n]")))))
    (regexp ": ?")
    (group-n 2 (regexp "[0-9]+"))
    (? (| (: "-"
             (group-n 4 (regexp "[0-9]+"))
             (? "." (group-n 5 (regexp "[0-9]+"))))
          (: (in ".:")
             (group-n 3 (regexp "[0-9]+"))
             (? "-"
                (? (group-n 4 (regexp "[0-9]+")) ".")
                (group-n 5 (regexp "[0-9]+"))))))
    ":"
    (| (: (* " ")
          (group-n 6 (| "FutureWarning"
                        "RuntimeWarning"
                        "Warning"
                        "warning"
                        "W:")))
       (: (* " ")
          (group-n 7 (| (regexp "[Ii]nfo\\(?:\\>\\|rmationa?l?\\)")
                        "I:"
                        (: "[ skipping " (+ ".") " ]")
                        "instantiated from"
                        "required from"
                        (regexp "[Nn]ote"))))
       (: (* " ")
          (regexp "[Ee]rror"))
       (: (regexp "[0-9]+?") ;; (regexp "[0-9]?")
          (| (regexp "[^0-9\n]")
             eol))
       (regexp "[0-9][0-9][0-9]"))))

SHOW YOUR WORK

One of my old math teachers used to say always "SHOW YOUR WORK". I came up with this guess-timated answer by first placing a message within the function compilation-parse-errors, with an eye towards extracting the regexp used to process the relevant components of the working data, which yielded the following regexp:

"^ *\\(?:[[:alpha:]][-[:alnum:].]+: ?\\|[ \t]+\\(?:in \\|from\\)\\)?\\(?1:\\(?:[0-9]*[^0-9\n]\\)\\(?:[^\n :]\\| [^-/\n]\\|:[^ \n]\\)*?\\)\\(?:: ?\\)\\(?2:[0-9]+\\)\\(?:-\\(?4:[0-9]+\\)\\(?:\.\\(?5:[0-9]+\\)\\)?\\|[.:]\\(?3:[0-9]+\\)\\(?:-\\(?:\\(?4:[0-9]+\\)\.\\)?\\(?5:[0-9]+\\)\\)?\\)?:\\(?: *\\(?6:\\(?:FutureWarning\\|RuntimeWarning\\|W\\(?::\\|arning\\)\\|warning\\)\\)\\| *\\(?7:[Ii]nfo\\(?:\>\\|rmationa?l?\\)\\|I:\\|\[ skipping \.+ ]\\|instantiated from\\|required from\\|[Nn]ote\\)\\| *\\(?:[Ee]rror\\)\\|[0-9]?\\(?:[^0-9\n]\\|$\\)\\|[0-9][0-9][0-9]\\)"

Then, I took the working data and used M-x re-builder to see what the above-mentioned regexp matched. I modified the second line of the working data by reducing the 14 to just one digit and that helped me zero in on the relevant section of the regexp at issue. From there, I looked at the compilation-error-regexp-alist-alist to locate the correspondenting section and found it in the gnu section of that variable.

12
  • Converting [0-9]? to [0-9]+? looks wrong. Surely it should be [0-9]*. (I.e. zero-or-more instead of the original zero-or-one.) I'm not sure what that number is, mind you, but I'm assuming we want to retain the support for it not existing at all, and I'm dubious that non-greedy behaviour could be correct.
    – phils
    Commented Jul 24, 2020 at 2:11
  • Most likely this needs an upstream bug report.
    – phils
    Commented Jul 24, 2020 at 2:12
  • @phils -- thank you for helping me to improve upon this answer. I used re-builder just now and see that [0-9]+ matches any integer (e.g. 5, 14, 100, 1000, etc.), and Emacs uses the compilation-line-number to colorize both the numbers 5 and 14 in each of the respective lines of working data. Could you give me an example, please, of when Emacs would be looking for something other than an integer (line number) in this example?
    – lawlist
    Commented Jul 24, 2020 at 2:22
  • I don't know for sure what this should be. The comments for the definition of the gnu section of compilation-error-regexp-alist-alist indicates that the expected format is PROGRAM:SOURCE-FILE-NAME:LINENO: MESSAGE but even the original working example ...Makefile.inc:5:0 (41): no match found... doesn't fit that. My guess was that :5:0 was :LINE:COLUMN and that COLUMN was intended to be optional, but at a glance I'm honestly unsure what is being matched.
    – phils
    Commented Jul 24, 2020 at 2:29
  • I have also assumed that this part of the regexp should only match numbers, though, and my suggestion was not at odds with that. I was just ensuring that, as with the original value, it was valid for it to be empty.
    – phils
    Commented Jul 24, 2020 at 2:31
1

This isn't a complete answer, but it provides more context.

As @lawlist has shown, this matching is determined by the gnu regexp in compilation-error-regexp-alist-alist, which is currently defined as follows:

    (gnu
     ;; The first line matches the program name for

     ;;     PROGRAM:SOURCE-FILE-NAME:LINENO: MESSAGE

     ;; format, which is used for non-interactive programs other than
     ;; compilers (e.g. the "jade:" entry in compilation.txt).

     ;; This first line makes things ambiguous with output such as
     ;; "foo:344:50:blabla" since the "foo" part can match this first
     ;; line (in which case the file name as "344").  To avoid this,
     ;; the second line disallows filenames exclusively composed of
     ;; digits.

     ;; Similarly, we get lots of false positives with messages including
     ;; times of the form "HH:MM:SS" where MM is taken as a line number, so
     ;; the last line tries to rule out message where the info after the
     ;; line number starts with "SS".  --Stef

     ;; The core of the regexp is the one with *?.  It says that a file name
     ;; can be composed of any non-newline char, but it also rules out some
     ;; valid but unlikely cases, such as a trailing space or a space
     ;; followed by a -, or a colon followed by a space.
     ;;
     ;; The "in \\|from " exception was added to handle messages from Ruby.
     ,(rx
       bol
       (? (| (regexp "[[:alpha:]][-[:alnum:].]+: ?")
             (regexp "[ \t]+\\(?:in \\|from\\)")))
       (group-n 1 (: (regexp "[0-9]*[^0-9\n]")
                     (*? (| (regexp "[^\n :]")
                            (regexp " [^-/\n]")
                            (regexp ":[^ \n]")))))
       (regexp ": ?")
       (group-n 2 (regexp "[0-9]+"))
       (? (| (: "-"
                (group-n 4 (regexp "[0-9]+"))
                (? "." (group-n 5 (regexp "[0-9]+"))))
             (: (in ".:")
                (group-n 3 (regexp "[0-9]+"))
                (? "-"
                   (? (group-n 4 (regexp "[0-9]+")) ".")
                   (group-n 5 (regexp "[0-9]+"))))))
       ":"
       (| (: (* " ")
             (group-n 6 (| "FutureWarning"
                           "RuntimeWarning"
                           "Warning"
                           "warning"
                           "W:")))
          (: (* " ")
             (group-n 7 (| (regexp "[Ii]nfo\\(?:\\>\\|rmationa?l?\\)")
                           "I:"
                           (: "[ skipping " (+ nonl) " ]")
                           "instantiated from"
                           "required from"
                           (regexp "[Nn]ote"))))
          (: (* " ")
             (regexp "[Ee]rror"))
          (: (regexp "[0-9]?")
             (| (regexp "[^0-9\n]")
                eol))
          (regexp "[0-9][0-9][0-9]")))
     1 (2 . 4) (3 . 5) (6 . 7))

That alist is preceded with the comment:

;; If you make any changes to `compilation-error-regexp-alist-alist',
;; be sure to run the ERT test in test/lisp/progmodes/compile-tests.el.
;; emacs -batch -l compile-tests.el -f ert-run-tests-batch-and-exit

The current gnu test cases from compile-tests.el are:

;; gnu
foo.c:88: message
../foo.c:88: W: message
/tmp/foo.c:88:warning message
foo/bar.py:8: FutureWarning message
foo.py:88: RuntimeWarning message
foo.c:88:I: message
foo.c:88.23: note: message
foo.c:88.23: info: message
foo.c:88:23:information: message
foo.c:88.23-45: Informational: message
foo.c:88-23: message
;; The next one is not in the GNU standards AFAICS.
;; Here we seem to interpret it as LINE1-LINE2.COL2.
foo.c:88-45.37: message
foo.c:88.23-9.17: message
jade:dbcommon.dsl:133:17:E: missing argument for function call
G:/cygwin/dev/build-myproj.xml:54: Compiler Adapter 'javac' can't be found.
file:G:/cygwin/dev/build-myproj.xml:54: Compiler Adapter 'javac' can't be found.
{standard input}:27041: Warning: end of file not at end of a line; newline inserted
boost/container/detail/flat_tree.hpp:589:25:   [ skipping 5 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]

to which we can add the two cases from this question (I've truncated the file paths, as that makes no difference).

Makefile.inc:5:0 (41): no match found, expected: ":", [ \t] or [\p{Latin}-_.${}/%0123456789]
Makefile.inc:14:24 (268): rule include_dir: include

We can then test these with M-x re-builder either by switching it to rx mode to use the original form1, or for the default read mode using (cadr (assoc 'gnu compilation-error-regexp-alist-alist)):

"^\\(?:[[:alpha:]][-[:alnum:].]+: ?\\|[ \t]+\\(?:in \\|from\\)\\)?\\(?1:\\(?:[0-9]*[^0-9\n]\\)\\(?:[^\n :]\\| [^-/\n]\\|:[^ \n]\\)*?\\)\\(?:: ?\\)\\(?2:[0-9]+\\)\\(?:-\\(?4:[0-9]+\\)\\(?:\\.\\(?5:[0-9]+\\)\\)?\\|[.:]\\(?3:[0-9]+\\)\\(?:-\\(?:\\(?4:[0-9]+\\)\\.\\)?\\(?5:[0-9]+\\)\\)?\\)?:\\(?: *\\(?6:\\(?:FutureWarning\\|RuntimeWarning\\|W\\(?::\\|arning\\)\\|warning\\)\\)\\| *\\(?7:[Ii]nfo\\(?:\\>\\|rmationa?l?\\)\\|I:\\|\\[ skipping \\.+ ]\\|instantiated from\\|required from\\|[Nn]ote\\)\\| *\\(?:[Ee]rror\\)\\|[0-9]?\\(?:[^0-9\n]\\|$\\)\\|[0-9][0-9][0-9]\\)"

This confirms the issue: All of the original test cases match, but only one of the new cases matches.

As @lawlist identified, changing that [0-9]? makes a difference. If we change that to [0-9]* then all of the cases are now matched; however there's so much going on in this pattern that it's currently unclear to me whether or not that's the correct fix.

In the failure case:

Makefile.inc:14:24 (268): rule include_dir: include

The line number is 14, but it's the subsequent 24 which is failing to match the zero-or-one-digit [0-9]?. Reducing that to a single digit (as seen in the case which worked) means the original regexp matches the line. (Use C-cC-u to ensure re-builder picks up the change, if necessary.)

That [0-9]? dates back to commit 0ab31e4a9ff from 2006, and was part of a change intended to "rule out false positives due to time stamps":

we get lots of false positives with messages including times of the form "HH:MM:SS" where MM is taken as a line number, so the last line tries to rule out message where the info after the line number starts with "SS".


1 You'll need make the top-level sequence explicit. Refer to the discussion of this gotcha in https://emacs.stackexchange.com/a/5577/454

3
  • It's possible that this was bug 15944, but sadly the reporter never followed up the request for more information, so it never went anywhere.
    – phils
    Commented Jul 25, 2020 at 1:38
  • After looking through the regexp, it seems to me that what Emacs is expecting from these errors is that there would be an additional colon : after the :LINE:COLUMN pairing. I.e. :14:24 would be :14:24:, and consequently the earlier part of the regexp (the section preceding that ":" separator, which is looking for line numbers and columns) would match.
    – phils
    Commented Jul 25, 2020 at 2:52
  • I'll add add that :LINE.COLUMN: seems like it might be the more common syntax; but in either case that trailing colon is expected to be there.
    – phils
    Commented Jul 25, 2020 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.