Note: This is a very complicated solution. See How to use xref-find-references on Elisp files without "No references found for" errors for when I came to my senses and simplified things.
Problems and fixes
I found multiple things that I did that tripped me up:
I had an ~/ID file that is generated using ID Utils. xref, or
something beneath it, ...
xref provides UI for code navigation. The actual functionalities are provided by the backend. In you case, the backend is etags.
If you open TAGS file, its format is very simple. It has only the information of tag definition. So you can not find reference of tag.
So you got two solutions,
Install lsp-mode and some language server, hope they ...
Use https://github.com/technomancy/find-file-in-project (ffip).
What ~ means does not matter. ffip will try to find file in project root using the file name under cursor.