Is there a way to add display annotation to the buffer text so that it's searchable?

Question

Suppose I have a text data file with various data values which the computer can parse easily, but it's not easy to read for a human.

In order to make the file easier to read I'd add display only annotation automatically to the file, e.g. like this:

3,10,'test',...

becomes

id = 3, length = 10, title = 'test', ...

These field labels can be added with, e.g. the before-string property. This works fine.

But what if I'd also like to search the file with isearch, but I don't just want to search for values (e.g. 'test'), but also the field names (e.g. title)?

Currently, this does not work, because the before-string is not part of the buffer text, so it's not searchable by isearch.

Is there a way somehow to add such field names to the file for display purposes, so they are not part of the file, and also make it possible to search these display names with isearch?

Is there some property which makes it possible for isearch to consider these added texts too when searching?

Answer 1

If I understand correctly (based on your comment to this answer), you want to search within some text that has a given overlay property (before-string) with a particular value. That is, you want search to skip over all text that does not have that property with that value.

You can do this with library isearch-prop.el. See the online description here.

---

before-string is an overlay property, not a text property. Overlay properties are attached to buffer positions. Text properties are attached to characters in the buffer.

You apparently want to search for a sequence of chars in a buffer such that there is a particular overlay property value at the buffer position of each matched char.

You can do this with isearch-prop.el, using command isearchp-property-forward, which is bound to C-t during Isearch. (You also have command isearchp-property-forward-regexp, bound to C-M-t.)

The first time you use it:

1. You're prompted for whether to search the buffer areas that have a given text property value or a given overlay value, or both. At the prompt, you enter overlay, to search only overlays.

2. Then you're prompted for the overlay property. With the first example code below, you'd enter my-ov-prop.

3. Then you're prompted for the value of that overlay property. With the first example, you'd enter foobar to search the areas that have property my-ov-prop with value foobar. (Hit an empty RET when you're done entering values.)

4. Then type the text you want to find in those areas. This is regular Isearch, but restricted to the buffer areas that have that overlay with that value.

(let ((ov1  (make-overlay 137 165))
      (ov2  (make-overlay 210 217))
      (ov3  (make-overlay 346 366)))
  (overlay-put ov1 'my-ov-prop 'foobar)
  (overlay-put ov2 'my-ov-prop 'toto)
  (overlay-put ov3 'my-ov-prop 'foobar)
  (message "OV1: %S, OV2: %S, OV3: %S" ov1 ov2 ov3))

Same thing for overlay property before-string:

(let ((ov4  (make-overlay 6 18))
      (ov5  (make-overlay 22 35))
      (ov6  (make-overlay 44 58)))
  (overlay-put ov4 'before-string "tata")
  (overlay-put ov5 'before-string "ubub")
  (overlay-put ov6 'before-string "tata")
  (message "OV4: %S, OV5: %S, OV6: %S" ov4 ov5 ov6))

At the first prompt, enter overlay. At the second prompt enter before-string. At the third prompt enter the string "tata" (and then an empty extra RET).

Then search for whatever you want in the areas of text that have property before-string with value "tata".

(The reason for hitting the extra RET, to say you're done entering values for the property, is because some properties, in particular face, allow multiple values, which get merged. Property before-string uses only one value.)

To repeat the same search you need not go through the prompting again. Just use C-u C-t during Isearch. (But as always with Isearch, to be able to use a prefix arg you need to set option isearch-allow-prefix to non-nil.)

If you ever want to search the text that's outside your propertied areas, instead of inside, just use M-= ~, to toggle searching within/outside those zones.

---

Here's the doc string of isearchp-property-forward:

isearchp-property-forward is an interactive compiled Lisp function in isearch-prop.el.

(isearchp-property-forward ARG)

Isearch forward in text with a text property or overlay property.

That is, move to the next such property and search within it for text matching your input.

Bound to C-t during Isearch.

If isearchp-complement-domain-p is non-nil then move to the next zone that does not have the given property. (Use M-= ~ during Isearch to toggle this variable.) For example, this lets you search for text that is NOT displayed using a certain face or combination of faces.

With no prefix argument, or if you have not previously used an Isearch command that searches properties or applies them, you are prompted for the following:

• the property type (text, overlay, or text and overlay)
• the property (e.g., face, mumamo-major-mode)
• the property values (e.g., a list of faces, for property face)

If you have previously used such an Isearch property command, then a prefix arg means reuse the property type, property, and value from the last Isearch property command. This includes isearchp-property-* commands and commands such as isearchp-imenu*, isearchp-thing(-regexp), isearchp-regexp-context-search, and isearchp-put-prop-on-region.

Note that this means that you can simply use C-u C-t during ordinary Isearch in order to repeat the last property search.

The particular prefix arg controls the behavior as follows:

• Plain C-u: Reuse the last property type (overlay, text, or both).
• Positive prefix arg (e.g., C-9): Search only text properties.
• Negative prefix arg (e.g., C--): Search only overlay properties.
• Zero prefix arg (C-0): Search both text and overlay properties.

By default, an actual value of the property matches the value you specify if it is equal. Properties mumamo-major-mode and face (or font-lock-face) are exceptions.

• For mumamo-major-mode, you specify the major mode whose zones of text you want to search. The actual property value is a list whose car is the major mode symbol.

• For properties face and font-lock-face, you can pick multiple faces, using completion (hit RET with empty input to finish choosing). Text is searched that has a face property that includes any of the faces you choose. If you choose no face (empty input at the outset), then text with any face at all is searched.

NOTE: If you search zones of property face and the property values include font-lock faces, then you might want to first make sure the entire buffer has been fontified. You can do that using command isearchp-fontify-buffer-now.

NOTE: This command is available during normal Isearch, on key C-t. However, in order to be able to use a prefix arg within Isearch, you must set isearch-allow-scroll or isearch-allow-prefix (if available) to non-nil. Otherwise, a prefix arg exits Isearch.

---

If, instead of wanting to search propertied text, you wanted to search for some text that has certain text properties, then, starting with Emacs 27, you can use command text-property-search-forward. (Load library `text-property-search.el'.) But I think it works only for text properties, not for overlay properties.