Search and replace to merge or eliminate text lines

Question

A WinRAR command (rar vt -pmypassword -v myarc.part01.rar >source.txt) produces detailed information on files contained in .rar files or segments (e.g., filename.part01.rar). Here's an example for one such file:

       Name: Folder Name\Subfolder\File Name.txt
       Type: File
       Size: 213
Packed size: 112
      Ratio: 52%
   Modified: 2022-12-16 19:46:23,000000000
    Created: 2022-12-31 06:56:42,887343500
 Attributes: I.A....
     BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a
    Host OS: Windows
Compression: RAR 5.0(v50) -m2 -md=256M
      Flags: solid encrypted

In this case, source.txt contains a vast number of such entries. That is, for each of many files, I have 12 similar lines of data.

For each such file, I would like to convert those many lines into a single line containing the Name, Type, and BLAKE2 lines, caret-delimited. Sample output:

Name: Folder Name\Subfolder\File Name.txt^Type: File^BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a

Notes: (1) There are eight blank spaces before "Name" in source.txt, and the colons in the other rows in source.txt are vertically aligned with the colon in "Name:". Superfluous spaces will ideally be removed from the resulting line. (2) In some cases, it is "Type: Folder" instead of "Type: File." (3) For the record, those are BLAKE2sp hashes, not BLAKE2.

Unless there is an easier way in Emacs, I can use repeated commands like grep -v "pattern" source.txt > target.txt; mv target.txt source.txt to eliminate each type of unwanted line (e.g., all occurrences of the Modified line) throughout the file.

That leaves me with the three lines containing the desired information, like this:

       Name: Folder Name\Subfolder\File Name.txt
       Type: File
     BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a

My question is how to convert those to a single line. I was thinking I would replace all occurrences of "line break plus eight spaces plus Type:" to ^Type:, and then I would do something similar with all occurrences of "line break plus six spaces plus BLAKE2:".

I'm using GNU Emacs 28.1 (build 2, x86_64-w64-mingw32). For the search-and-replace operation that would put ^Type: onto the Name: line, I think I have the Search part working: Ctrl-S > Enter > Ctrl-W > Ctrl-Q > Ctrl-J > "        Type: " (i.e., eight spaces, then Type:, then another space, without quotation marks). But I can't figure out how to enter the Replace part.

Answer 1

I know that you are looking for an Emacs solution and it may be that somebody will provide one, but that's the wrong way to go about the problem: you should use the right tool for the job. There is more to life than Emacs alone.

In this case, awk provides "record processing" where a "record" is a sequence of lines. In order to mark the end of the record, awk assumes that records are separated by an empty line. So my suggestion is to regenerate your file and add that empty line between records in order to make life easier for subsequent processing.

Given an input file like that, the following awk script will (almost) do what you want:

BEGIN { RS = ""; FS="\n";}
      { printf "%s^%s^%s\n", $1, $2, $9; }

The initialization line says the record separator RS has the special value "" (the empty string). That is a signal to awk that the input file has the structure described above and each record is a sequence of lines. The field separator FS is set to newline, thus making each line into a separate field. So the Name line is field 1, the File line is field 2 and the BLAKE2 line is field 9.

The second line describes the processing (with an implicit loop over all the records as always in awk). It prints fields 1, 2 and 9, separated by carets. Assuming that the script is in a file script.awk, you run it like this:

awk -f script.awk -- input.txt

On an input consisting of two copies of the record you show in your question (with one copy separated by an empty line from the next), the script gives:

       Name: Folder Name\Subfolder\File Name.txt^       Type: File^     BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a
       Name: Folder Name\Subfolder\File Name.txt^       Type: File^     BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a

As you can see, what the script does not do is deal with the leading white space. awk has functions for manipulating strings, so it is conceivable to come up with one to strip the spaces, but there is a better way. We can use sed to get rid of leading white space, even before awk gets its hands on the input. Unix notates such chaining of programs with the output of one becoming the input of the next one using the pipe operator |. Here's a sed command to get rid of initial whitespace:

sed 's/^ *//' input.txt

That can then be combined in a pipe with awk running the script above to do the second part:

sed 's/^ *//' input.txt | awk -f script.awk

and you are done. The output looks like this:

Name: Folder Name\Subfolder\File Name.txt^Type: File^BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a
Name: Folder Name\Subfolder\File Name.txt^Type: File^BLAKE2: d00d0d72343d14fa2e6a4201d613df41702f8cd408527c435bb566c1cbda532a

I submit to you that this is a better way to do things than trying to combine Emacs interactive operations into a macro that would then have to be repeated maybe thousands of times. At the very least, it would be a slow way to deal with large inputs.