Library isearch-prop.el can help with at least some of what you want to do, I think.
See command isearchp-add-regexp-as-property or function isearchp-regexp-scan, for adding text properties based on regexps, to give you propertied zones amenable for searching.
Other commands and functions in the library let you search such propertied zones or their complements. For example, you can use C-t within Isearch (or C-M-t within regexp Isearch) to search within such a zone. (C-M-~ toggles searching the complement.)
This adds property foo (with value ("ref.+")) to text matching the regexp ref.+:
(isearchp-add-regexp-as-property (point-min) (point-max) 'foo)
(You can also apply other properties, such as face or mouse-face or invisible, to the same text, depending on whether you want some particular display behavior.)
But note that if your aim is just to propertize certain positions or zones of text so that you can quickly get to them later then, then unless you need to insert your refXXXX text for some other reason, you can skip that text insertion and just propertize the locations/zones directly. You can easily do that using isearch-prop.el.
IOW, why are you bothering to insert refXXXX text that you later want to hide (or highlight?), if that text is just meant to identify buffer positions or zones?
If you just want to mark positions, you can of course use markers, but they do not have identities (i.e., additional info) beyond their positions. If you want to mark the text itself, you can use properties to do that.
Here are the doc strings for the functions that add properties that let you search regexp-defined zones (or their complements):
(isearchp-regexp-scan BEG END PROPERTY REGEXP &optional PREDICATE ACTION)
Scan for REGEXP, adding text property PROPERTY to matches.
If the region is active then scan it. Otherwise scan the buffer.
Return the value of PROPERTY, if added somewhere. Else return nil.
See `isearchp-add-regexp-as-property' for the parameter descriptions.
(isearchp-add-regexp-as-property PROPERTY REGEXP &optional BEG END PREDICATE ACTION MSGP)
Add PROPERTY with value (REGEXP . PREDICATE) to REGEXP matches.
If region is active, limit action to region. Else, use whole buffer.
If REGEXP has subgroups, then use what the Nth subgroup matches as the
search context (hit), where N = `isearchp-context-level'.
If N=0, then use the overall match of REGEXP as the search context.
PROPERTY is a text property to add to the search-hit text.
the value of the property is REGEXP (a string).
Interactively, PREDICATE is nil, so the PROPERTY value is (REGEXP).
Non-interactively:
BEG, END are the region limits. If nil, the buffer limits are used.
PREDICATE is nil or a boolean function that takes these arguments:
- the search-context string
- a marker at the end of the search-context
If PREDICATE is non-nil then act on only the hits for which it holds.
If ACTION is non-nil then it is a function that accepts no arguments.
It is called after matching buffer text with REGEXP. After ACTION,
the search hit end position is extended or restricted to point.
Non-interactively, non-nil BEG and END are used as the region limits.
Library isearch-prop.el is a companion to library isearch+.el (neither is needed for the other, but they enhance each other). See also Isearch+.
