11

I am reading detailed line oriented log file (100MB).

I want to skip some parts by using regex.

Usually I M-x flush-lines but this is destructive operation.

I want to hide many lines by regex and expect it would be efficient when I navigate across file. Extra plus if interactive search skips hidden parts.

From the comments on yue's answer:

  • I want incremental exclusion (first regex, then next regex).

  • I search for anomalies in log and know what to exclude but I don't know what I will have found.

Improve this question
5
  • 2
    Why don't you like flush-lines? With a tiny trick you can clone the file buffer into one that does not visit any file and flush lines there. (1) Do you want a list of flushes you have performed to roll back? (2) Do you want to edit the log file (maybe inserting some kind of markers) and keep these edits in hidden regions? Please, expand your question in that regard.
    – Tobias
    Commented Apr 30, 2019 at 11:44
  • @Tobias It is exploration of log for some issues and it can be possible that I applied flush-lines too aggressively and losing problem. Probably should work on file copy...
    – gavenkoa
    Commented Apr 30, 2019 at 11:50
  • I.e. you want to be able to roll-back flush-lines commands in an undo-tree fashion maybe including the undo-tree-visualizer-toggle-diff feature. It would be even nicer to see the flushed regexps aside the undo tree nodes. Something like that? You can use all that on a buffer clone as I stated in my first comment.
    – Tobias
    Commented Apr 30, 2019 at 11:59
  • The mentioned strategy is even reasonably fast with large text files (just tried it with 40MB).
    – Tobias
    Commented Apr 30, 2019 at 12:06
  • 1
    I know the file is quite large, duplicating the file might not be optimal, but I ended up doing just that. It's just a few steps. C-x C-w, foo.log, M-x flush-lines. After that, you can do whatever you want with the duplicated file.
    – frogcoder
    Commented Nov 18, 2019 at 12:18

4 Answers 4

Reset to default
5

You want to use occur. This feature is built-in and it does exactly what you want.

There is also a package called loccur, which does the same, but does not create a new window (it just hides all non matching lines).

Improve this answer
5
  • I want incremental exclusion (first regex, then next regex). occur has positive logic instead of negative and I can't apply exclusion sequentially...
    – gavenkoa
    Commented Apr 30, 2019 at 11:05
  • I search for anomalies in log and know what to exclude.
    – gavenkoa
    Commented Apr 30, 2019 at 11:07
  • @gavenkoa not sure if I understand correctly, but you can run occur in an occur-buffer, then it filters from the already filtered results.
    – jue
    Commented Apr 30, 2019 at 12:30
  • @gavenkoa you could also run occur again in your logfile-buffer, recall the old regex (by using cursor key <up>) and edit the old regex
    – jue
    Commented Apr 30, 2019 at 12:39
  • occur in occur adds level of line numbers as prefix + it is additive logic instead of negative.
    – gavenkoa
    Commented Apr 30, 2019 at 13:35
5

I agree with comments that suggest using flush-lines. I would likely use that in this case, especially since your use case is line-oriented.

But if you want to make text that matches a regexp invisible, so that search, query-replace, etc. ignore it but it is still present, then you can do that with libraries Zones and Isearch+.

For example, you can use C-x n R (command zz-set-zones-matching-regexp) to set the value of the current izones variable to the list of zones that match a given regexp. Then use command isearchp-make-zones-invisible to make those zones invisible.

If you want to make other lines, which match other regexps, then use C-x n r, not C-x n R. The former adds zones to the zone set; the latter sets the zone set (removing any existing zones in it).

There are also these related commands: isearchp-make-zones-visible, isearchp-toggle-zones-invisible, and the same for anti-zones (complement of zones).

C-x n R and C-x n r match a regexp against your text, like re-search-forward does - the search hits define the zones. They are not line-oriented like flush-lines, occur, and grep. So if you want an entire line to become invisible then you need to use a regexp that matches the entire line (containing whatever text you're interested in).

Improve this answer
1

Try this:

M-x query-replace-regexp RET .*FOO.* C-qC-jRET \,(propertize \& 'invisible t) RET

See also C-hig (elisp)Invisible Text

Improve this answer
1

I found this most easy solution. https://github.com/vapniks/hide-lines

Some functions are here which can be useful for your scenario

hide-lines : Hide lines matching the specified regexp.
Keybinding: M-x hide-lines
hide-lines-not-matching : Hide lines that don’t match the specified regexp.
Keybinding: M-x hide-lines-not-matching
hide-lines-matching : Hide lines matching the specified regexp.
Keybinding: M-x hide-lines-matching
hide-lines-show-all : Show all areas hidden by the filter-buffer command.
Keybinding: M-x hide-lines-show-all
hide-blocks-not-matching Hide text that is not between lines matching START-TEXT and END-TEXT. Keybinding: M-x hide-blocks-not-matching
hide-blocks-matching Hide text that is between lines matching START-TEXT and END-TEXT. Keybinding: M-x hide-blocks-matching
hide-lines-kill-hidden Kill or delete all hidden areas. Keybinding M-x hide-lines-kill-hidden
Improve this answer
1
  • I need to evaluate this solution because flush-lines is quite performant on jumble files (like 200MB of text logs).
    – gavenkoa
    Commented Jan 20, 2021 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.