0

I am using GNU Emacs 26.3 on Ubuntu 19.10.

I have a directory full of Emacs Lisp files with the .el file extension. All of them are loaded into the currently running Emacs. I can open up one of those files, put point on a function that is being called, and then use M-. to go to that definition. That works. But when I use M-? (which is bound to xref-find-references) I get an error of the form:

user-error: No references found for: xxx

where xxx was the symbol at point.

I thought this was due to not using a TAGS file, so I generated one using:

cd ~/my_emacs_lisp_dir1
etags -l lisp ../my_emacs_lisp_dir1/*.el ../my_emacs_lisp_dir2/*.el

And that did work as it produced a TAGS file where I expected it to be which is in ~/my_emacs_lisp_dir1/TAGS.

In Emacs, I visited that TAGS file using visit-tags-table. And then tried M-? on the symbol again, and it still failed with the above error.

I then executed xref-etags-mode from within that same .el buffer and tried again. Still I got the same error.

Opening up the TAGS file and searching for the symbol only turns up what I believe to be an indication of which file a symbol is defined in, but I don't see how it could record references to where that symbol is referenced from. So that may be the problem, and if so, how do I make it tell me which functions and files refer to the symbol at point?

Update 1: It works for standalone files, but not in my Git-controlled HOME directory

I noticed that it does work if I do something silly like this:

rm -rf /tmp/xref-test-dir
mkdir -p /tmp/xref-test-dir
cd /tmp/xref-test-dir


cat > func-1.el <<'EOF'
(defun xxx-func-1 ()
  (xxx-func-2))
EOF

cat > func-2.el <<'EOF'
(defun xxx-func-2 ()
  (message "xxx-func-2 called"))
EOF

etags -l lisp *.el

Then go to each of the two .el files, and run eval-buffer on them and then point point on the call to xxx-func-2 and it at least did something that I think is correct: It showed a buffer with this info:

/tmp/xref-test-dir/func-1.el
2:   (xxx-func-2))
/tmp/xref-test-dir/func-2.el
1: (defun xxx-func-2 ()
2:   (message "xxx-func-2 called"))

I noticed that somewhere along the way I was prompted for a project directory and I supplied it /tmp/xref-test-dir.

But I got no such prompting from when I did it inside my ~/my_emacs_lisp_dir1 directory.

I dug even deeper into the xref-backend-references function, and found some code dealing with projects. So I went back to the .el file inside ~/my_emacs_lisp_dir1 directory, and ran this via eval-expression:

(project--find-in-directory ".")

and it returned this:

(vc . "~/")

that sort of makes sense because both ~/my_emacs_lisp_dir1 and ~/my_emacs_lisp_dir2 are within a Git repo.

But I'm still stuck because I don't know how to tell "it" to just look into those two directories, or even look into the files in that same directory.

  • 1
    Are the directories in question inside load-path? xref-backend-references searches the "current project" and the external roots, which in this case are the list returned by elisp-load-path-roots, which see. – Dmitry Mar 17 at 22:16
  • @Dmitry Yes that turned out to be the case for sure. But there was more to it than that. See my answer at emacs.stackexchange.com/a/56281/15483 – bgoodr Mar 21 at 23:01
1

xref provides UI for code navigation. The actual functionalities are provided by the backend. In you case, the backend is etags.

If you open TAGS file, its format is very simple. It has only the information of tag definition. So you can not find reference of tag. enter image description here

So you got two solutions,

Solution 1, Install lsp-mode and some language server, hope they will have implementation for finding references.

Solution 2, grep project with other packages. You can try counsel-git-grep or counsel-rg.

You can read Emacs Lisp code by yourself to confirm my point.

By default, xref--find-xrefs is called. Its definition,

(funcall (intern (format "xref-backend-%s" kind))
                        (xref-find-backend)
                        arg)

kind is references, so xref-backend-references is called. Here's its definition,

(cl-defgeneric xref-backend-references (_backend identifier)
  "Find references of IDENTIFIER.
The result must be a list of xref objects.  If no references can
be found, return nil.

The default implementation uses `semantic-symref-tool-alist' to
find a search tool; by default, this uses \"find | grep\" in the
`project-current' roots."
  (cl-mapcan
   (lambda (dir)
     (xref-collect-references identifier dir))
   (let ((pr (project-current t)))
     (append
      (project-roots pr)
      (project-external-roots pr)))))

Notice the parameter _backend is ignored. I'm not sure what's the value of semantic-symref-tool-alist, maybe it's set up by you or the 3rd party packages you installed.

Definition of semantic-symref-tool-alist,

(defvar semantic-symref-tool-alist
  '( ( (lambda (rootdir) (file-exists-p (expand-file-name "GPATH" rootdir))) .
       global)
     ( (lambda (rootdir) (file-exists-p (expand-file-name "ID" rootdir))) .
       idutils)
     ( (lambda (rootdir) (file-exists-p (expand-file-name "cscope.out" rootdir))) .
       cscope )
    )
  "Alist of tools usable by `semantic-symref'.
Each entry is of the form:
   ( PREDICATE . KEY )
Where PREDICATE is a function that takes a directory name for the
root of a project, and returns non-nil if the tool represented by KEY
is supported.

If no tools are supported, then 'grep is assumed.")

So you could find real references if and only if you install GNU Global or GNU ID Utils or Cscope.

The documentation does mention the grep will be used as fallback. But I tested with a real project. The grep didn't happen.

| improve this answer | |
  • semantic-symref-tool-alist falls back to Grep if no other databases are found, so your last sentence is false. It should "just work". As long as the references are in files belonging to tags-table-list (if xref-etags-mode is enabled) or inside load-path. – Dmitry Mar 18 at 16:54
  • @Dmitry, according to the documentation, you are right. But I tested with real project (I provided the project root) , seems the grep doesn't happen. – chen bin Mar 19 at 0:55
  • Could you file a bug report with an example? If you test it first with Emacs 27, that would be great. – Dmitry Mar 19 at 14:07
  • @chenbin Your answer was helpful and instrumental in me arriving at the answer at emacs.stackexchange.com/a/56281/15483 . "So you could find real references if and only if you install GNU Global or GNU ID Utils or Cscope." is not actually true in my case because I was able to use "the grep backend" as long as I forced the elisp code to not add all of load-path to the directories it searches, for performance reasons. – bgoodr Mar 21 at 23:03
1

Note: This is a very complicated solution. See How to use xref-find-references on Elisp files without "No references found for" errors for when I came to my senses and simplified things.

Problems and fixes

I found multiple things that I did that tripped me up:

  1. I had an ~/ID file that is generated using ID Utils. xref, or something beneath it, was trying to use ID as its backend, because it is much, much faster than grep, of course. However, the way I have to generate these ID files is by disabling all language specific parsing in the lang-map file I use with gid (see the mkid lang-map below). But mkid for text treats dashes as punctuation, and thus, not a part of a symbol name. And, most if not all of the Emacs Lisp symbols I define have dashes. Thus, when I was searching using M-?, it used lid to find them and got nothing. Even when I called mkid with a lang-map that marked *.el files as lisp, something still wanted to search all of ~/ which is not what I need (see next item).

  2. Removing the ID files caused it to work, but it took quite a bit longer time than I know it should if it was only to search my small bit of Elisp code. It turns out that the slowness was caused by the elisp definitions that include everything in the load-path which is a ton of directories. So that had to be changed to limit the search to only one directory (containing subdirectories). It is probably correct to do this by default (if you intend to search for everything that Emacs knows about, which is useful in some contexts), however, not in this special case where I have only two directories.

My ~/elisp/.dir-locals.el file

;; -*- fill-column: 100 -*-
;;
;; This is needed so that M-? (runs the command xref-find-references) can not trip over ID files
;; (created by my use of mkid from the idutils package).  When I create these ID files, I cannot
;; specify things for Elisp that uses "-"s in the symbol names, because I use gid to find things
;; everywhere, even text files. When searching for an Elisp symbol that contains dashes, it cannot
;; find it inside the .el files referenced by that ID file. So the ID backend has to be deactiviated
;; for searching purposes for M-?.
;;
((emacs-lisp-mode . ((eval . (let ((my-elisp-dir "~/elisp"))
                   ;;
                   ;; By default, the "vc" part of the xref code finds ~/.git and thus
                   ;; assumes the entire Git-controlled directory, ~/, is to be
                   ;; searched. But that results is a HUGE amount of search time
                   ;; (because grep is used, and not something like ID or Global etc.)
                   ;; because ~/ also has ~/emacs.d/ which houses all of the packages I
                   ;; use. Instead, we have to force that "off" by forcing it to only
                   ;; look inside my-elisp-dir.
                   ;;
                   (eval `(defun my-elisp-project-find (dir)
                    (cons 'vc
                          ;; Use file-name-as-directory to avoid the assert inside
                          ;; xref-collect-references:
                          ,(file-name-as-directory my-elisp-dir))))
                   ;;
                   ;; Make project-find-functions buffer-local as this should not apply
                   ;; anywhere else but here:
                   ;;
                   (make-variable-buffer-local 'project-find-functions)
                   (add-hook 'project-find-functions #'my-elisp-project-find)
                               ;;
                               ;; Do not let the xref code search all of the other directories in
                               ;; the elisp "load-path", which tends to be huge, as it includes both
                               ;; the site-lisp area in the installed Emacs *and* all of the stuff
                               ;; inside ~/emacs.d from Elpa etc. I intend to be searching for my
                               ;; personally-defined Elisp functions when I'm doing my own personal
                               ;; development.
                               ;;
                   (setq-local project-vc-external-roots-function
                       `(lambda ()
                          (mapcar
                           ;; Something is buggy in some lower level emacs lisp
                           ;; code that does not accept "~" in the paths. So
                           ;; hack around that bug with expand-file-name:
                           #'expand-file-name
                           (list ,my-elisp-dir)))))))))

My mkid lang-map

# Welcome to the mkid language mapper.
#
# The format of each line is:
#
#       <pattern> <language> [options]
#
# Filenames are matched top-to-bottom against the patterns, and the
# first match is chosen.  The special language `IGNORE' means that
# this file should be ignored by mkid.  The options are
# language-specific command-line options to mkid.
#
# If a file name doesn't match any pattern, it is assigned the default
# language.  The default language may be specified here with the
# special pattern `**', or overridden from the mkid command-line with
# the `--default-lang=LANG' option.
#
# The special pattern `***' means to include the named file that
# immediately follows.  If no file is named, then the default system
# language mapper file (i.e., this file) is included.

# Default language
**          text

# Ignore PDF files as they are binary and don't provide much value in
# the output, and cause mkid to chew up CPU:
*.pdf           IGNORE
| improve this answer | |
  • 1
    You have an unusual setup. Can't you have idutils scan Elisp in the normal way, though? I think it supports Emacs Lisp the language. – Dmitry Mar 22 at 12:17
  • @Dmitry Yes, I can use mkid with a lang-map file that enables "lisp" parsing on .el files. I have .org files comingled in the same directories as the ones containing the .el files. Sure, I could have a idutils lang-map that calls out .org files as text and .el files as lisp. And I might try that someday. However, since I cannot store lang-map files in situ with all of the directories I need to run mkid on, I am stuck with a one-size-fits-all lang-map for now until I find a smarter way to manage it all (contd.) – bgoodr Mar 23 at 1:26
  • (cont.d) Furthermore, the reason I have a one-size-fits-all mkid lang-map file is that I also needed to run mkid on a directory that contains a huge amount of files with varying file extensions and treat them all as what mkid calls "text", even though that is not true. For instance a huge amount of the files being indexed in this huge directory are source code (none of it being (Emacs) Lisp) and several other languages, of which there is "cross-pollination" one language "calls" another language, with the same symbol. Since the calls are in (contd.) – bgoodr Mar 23 at 1:29
  • 1
    OK. Maybe try it again in Emacs 27 when it comes out. – Dmitry Mar 25 at 20:44
  • 1
    @Dmitry You were right the first time above. I finally came to my senses and added back a ID file right alongside my ~/.git directory and fed mkid a langmap that treats *.el files as lisp language. Now M-? works nicely. – bgoodr Mar 29 at 18:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.