How to font lock the second token on a line of tokens

Question

I'm trying to font lock some tokens in a simple text file, without picking up adjacent tokens. For example, here are some example lines that I have been working with:

name: foobar (&optional arg1 arg2)
key:  somechars-1
(defun my-fun-name

I want to recognize and font lock the second token on each line -- "foobar", "somechars-1", or "my-fun-name", without matching "name:", "arg1", or "key:". I've tried using re-builder and a bunch of regexp examples from the net, but without success.

I've tried what I know from using font-lock-defaults in major/minor modes, but what I know is not doing the job.

It's easy to match and font lock tokens with unique syntaxes (like pathnames), or to match the whole line, or both the key: and first token of the value. But I can't figure out how to match only the second token on each line, when there are other (optional) tokens on the same line, with the same syntaxes.

Various language modes do this (eg Lisp mode font locks the defun names), but I don't know how they do it. I looked at the lisp-mode code with pcase-let matching patterns, and could see them matching keywords, error tokens, etc, but I couldn't recognize anything that showed how it matches the second token on a line.

Perhaps I am using the wrong approach, thinking in a "font-lock-defaults" way for a major mode. Those kinds of regexp matches are global kinds of things. Maybe the only way to do what I want is to parse the lines, mark off the special tokens programmatically, and font lock them individually. I don't know the right answer (or maybe even the right approach).

Perhaps someone could explain how to do this with a short example, or point me to some kind of a reference doc that explains how to do this kind of font locking matching. (or how to match the nth token on a line, when all the syntaxes are the same (eg foobar and arg1 in my example above). Thank you

Answer 1

The following regex will do what you want for your examples:

^[ \t]*[^ \t\n]+[ \t]+\\([[:alpha:]][^ \t\n]*\\)

It matches the start of line ^. Then possible leading whitespaces, everything but whitespace [ \t]*[^ \t\n]+. Then at least one whitespace [ \t]+. Now it matches in a capturing group anything but whitespace or carriage return, starting with a letter \\([[:alpha:]][^ \t\n]*\\) (this depends a bit on what exactly a valid token is in your case).

So you can use group 1 to font lock:

(font-lock-add-keywords nil '(("^[ \t]*[^ \t\n]+[ \t]+\\([[:alpha:]][^ \t\n]*\\)"
                               1
                               'font-lock-keyword-face)))

Note that I intentionally used [ \t] and not \\s-, because the regexp would otherwise also match newline and become instable.

EDIT:

The following example shows how to fontify more matching groups:

(font-lock-add-keywords nil '(("^[ \t]*\\([^ \t\n]+\\)[ \t]+\\([[:alpha:]][^ \t\n]*\\)"
                               (1 'font-lock-keyword-face)
                               (2 'font-lock-variable-name-face))))

By this a regexp matching several part of interest must be evaled only once to fontify each part. If course the first group would have to be more specific to be really usefully fontified, probably something like "^[ \t]*(?\\([[:alpha:]][[:alnum:]_-]+\\):?[ \t]+\\([[:alpha:]][^ \t\n]*\\)".