make projectile search functions work for non project files? (or include them as project files?)

Question

Below is the situation I'm trying to solve

project root is at ~/project/src/some.py

`xref-find-definition` from `some.py` can go to 

python library in ~/.virtualenvs/myenv/lib/another.py

Then, since ~/.virtualenvs/myenv is not part of the project, I'm unable to do any more project search such as projectile-ag

Related yet slightly different situation occurs when .emacs.d/.gitignore has elpa as ignored directory. But I want to search within elpa directory.
Here since elpa is one of child directory of the parent directory, the situation is slightly different.

** edit

(xref-find-backend) gives me eglot.

So eglot knows more (it can understand the importing file from outside of the project) than projectile (which is directory based, often times .git)

So then, instead of using git-grep, ag, rg, why can't we just use xref-find-something? I guess xref only cares about the whole symbol not the partial text, so it doesn't have notion of "grep"ping partial text.

So then, question becomes.
(and this is my assumption) Since xref or eglot, any backend knows about the whole structure including the imported library.

I wonder if I can extract the info (which folders or which files are imported into this current project) from the backend, and give it to projectile, so that ag, rg, grep can be used on the folders/files.

Or if that's too hard, I can do it manually if projectile allows me to add any directory outside of the project root directory.

Or, can TAGS file help? since those library files are not changed often, I can generate the TAGS for the library files.

Would I be able to search

• my source using something like git-grep which reads the current file
• and library files using TAGS which were built previously

which are both recognized by projectile?
Practically I just need to bind single key to do this, and it is easier to do it if I can make something (such as projectile) to be aware that they belong to the same project.

** 2nd edit

So there are two different functionalities.

(find definitions) xref-*

• it's handled by eglot-extend-to-xref
• lsp backend's capability or setup (certain backend allows you to add extra directories outside of project)

(textual grep) rg/ag/git-grep

• it's handled by projectile, helm for my setup and
• project based searches are limited by projectile (not sure if you can add extra directories outside of project-root, but it doesn't seem possible)
• and it is also limited by the search tool (rg/ag/git-grep), git-grep won't be able to find non git-managed files.

The best I can do at the moment is..

• setup eglot / lsp backend so that xref-* can find relavant sources outside of project
• once you land on files outside of files, mark the root of the library as a project
  • using .projectile if you use file based search such as rg/ag
  • using .git if you use git-grep

Then you can do xref-* searches and textual searches. But it's hard to do textual grep on both directories, they all seem to assume single root directory to search on.

• TAGS might be able to handle it but it takes time to keep it updated.

Answer 1

If you didn't generate a TAGS file manually (and you probably didn't), Projectile doesn't have anything to do with xref-find-definitions.

You should find your which Xref backend is being used in your Python buffers (e.g. by evaluating (xref-find-backend)), and then probably direct the question to the third-party package which provided this backend.

EDIT: Eglot has a user option eglot-extend-to-xref which is off by default (I don't quite remember the reasoning, but one could search the issues). If you customize it to t, it will likely give you the behavior you wanted (tagging external files as belonging to a project during navigation).