Is there a way to attach hidden/invisible data to a place in text?

Question

I want to keep track of some stuff happening in the buffer, with elisp, and display indicators depending on that stuff. To that end, I need to cache calculated data for different parts of the file, and then quickly access and update that data. Specifically, this will be going on in Org-mode, and my data is associated with individual headings.

The conundrum here is, how do I best store this data, in memory, so it's associated with headings but not visible and not written to the file? I.e. when I insert or remove text in one place, points later in the file need to keep data associated with them—I can't just use text positions as keys in a dictionary (not without lots of hassle). It's also preferable that hidden data is not copied around together with yanked text.

To clarify, I'm aware that I can add properties on Org headings, or create IDs and associate my data with them—but I want to do this without any changes to the content.

So far it seems that ‘text properties’ might vaguely be what I want—if I can add my own custom fields in properties on headings. Elisp documentation notes that these properties are copied when text is yanked, and suggests putting them on an overlay instead. I'm rather hazy yet on how overlays function at all (though will probably need them for the indicators). Is this the best route, or maybe I'm looking in an entirely wrong place? Any nice rakes here for me to step on or shoot my feet?

Meanwhile, ‘invisible text’, popular in web search results, is not the thing I need—as afaict it's written to the file.

Answer 1

John Kitchin wrote a package called ov-highlighter that makes highlighting persistent. IIRC, it has a hook that update after each file saving a local variable storing a list of region and propertized string. I suggest you to read his post, then you can move to the source code to read the two relevant functions ov-highlight-load and ov-highlight-save to see how that is done.

A second idea may be using something like l3kn/org-el-cache to cache Org mode data. I didn't try it, but AFAIK it is used in l3kn/org-zettelkasten.

Answer 2

I believe you're looking for "overlays":

You can use overlays to alter the appearance of a buffer’s text on the screen, for the sake of presentation features. An overlay is an object that belongs to a particular buffer, and has a specified beginning and end. It also has properties that you can examine and set; these affect the display of the text within the overlay.