How to delete numerals of a certain size from text

Question

I'm pretty new to emacs, so I'm still learning all the functions.

I'm trying to remove certain metadata from each line of several large .txt files (Bible translations) and was wondering if if there was a way to search for and delete numerals above a certain value or to delete a specific column (the 4th) separated by TAB characters.

BOOK <TAB> CHAPTER# <TAB> VERSE# <TAB> VERSE-ID# <TAB> TEXT

The 4th field is a universal verse number starting at "10" and increases sequentially by 10s, so it becomes quite large - far larger than any other numeral in the file, which would otherwise be less than 300.

An example of what I'd like to do would be changing this:

PSALMS  150 5       164000  Praise him upon the loud cymbals: praise him upon the high sounding cymbals.
PSALMS  150 6       164010  Let every thing that hath breath praise the LORD. Praise ye the LORD.

to this:

PSALMS  150 5   Praise him upon the loud cymbals: praise him upon the high sounding cymbals.
PSALMS  150 6   Let every thing that hath breath praise the LORD. Praise ye the LORD.

The files also are divided into books by org-mode headings:

* THE NEW TESTAMENT
** The GOSPEL According to Matthew
MATTHEW 1   1       231460  The book of the generation of Jesus Christ, the son of David, the son of Abraham.

I suppose a delete-by-column function would have to be iterated, but that's not an issue for me as I can easily whip up a macro. Messing up the formatting (i.e. leaving an empty field) should be easy enough for me to clean up, so simply either deleting the column or an arbitrarily large numeral is acceptable.

Answer 1

Since you have a very specific line format, we can use regexes to accomplish this task. This method will go through your file, and on each line that is in that specific format, replace it with the line without that section.

I've left it inserting tabs; if you would rather have spaces, the replacement (the argument to replace-match) can be easily modified.

(defun remove-verse-ids ()
  (interactive)
  (goto-char (point-min))
  (while (re-search-forward "\\([[:alnum:]]*\\)\t\\([[:digit:]]*\\)\t\\([[:digit:]]*\\)\t[[:digit:]]*\t\\(.*\\)"
                            nil
                            t)
    (replace-match "\\1\t\\2\t\\3\t\\4")))

The regex used in re-search-forward is a little wordy; but I'm not sure what would easily shorten it, other than replacing [[:digit:]] with [0-9], which I believe would be slightly less readable.

For fun, an explanation of the regex follows. All lines beginning with ;; are comments as explanations.

\\([[:alnum:]]*\\)
;;First, any number of alphanumeric characters, captured for later use. The "book".

\t
;;A tab character.

\\([[:digit:]]*\\)
;;Any number of, well, numbers, captured for later use. The chapter number.

\t
;;A tab character.

\\([[:digit:]]*\\)
;;Any number of numbers, captured for later use. The verse number.

\t
;;A tab.

[[:digit:]]*
;;Numbers. The verse ID number, which is not captured.

\t
;;Tab

\\(.*\\)
;;Any number of characters, for later use. The actual text.

Answer 2

... a way to search for and delete numerals above a certain value

I think column-based solutions make more sense in this case but, just for fun, here's an interactive way you could delete numerals above a certain value using search-and-replace:

M-x replace-regexp RET \b\([0-9]\{6,\}\)\b RET \,(if (> \#1 164000) "" \1) RET

Which will turn your example into:

PSALMS  150 5       164000  Praise him upon the loud cymbals: praise him upon the high sounding cymbals.
PSALMS  150 6         Let every thing that hath breath praise the LORD. Praise ye the LORD.

By changing numbers greater than 164000 to an empty string.

This is matching only numbers of at least six digits, and then testing all matched numbers to see whether they're > 164000, and replacing them with either the original matched text OR an empty string, depending on the test.

---

As you've said you're new to Emacs, I'll add that this is a somewhat advanced technique, combining regular expressions, emacs lisp, and some special-purpose interactive replacement syntax; but you might find it interesting to figure out how that all fits together. You can also use query-replace-regexp instead of replace-regexp in order to step through the replacements one at a time, which will show the process a bit better.

Answer 3

A way to do it interactively:

M-x query-replace-regexp ^\([A-Z]+\)\([  ]+\)\([0-9]+\)\([   ]+\)\([0-9]+\)[     ]+[0-9]+\([     ]+\)\(.+\) RET \1\2\3\4\5\2\7 RET

For those not familiar with regular expressions: [A-Z] matches all upcase characters, [0-9] matches integers, the "+"-flag means: one-or-many occurrences, the \(..\) grouping construct permits to fetch the stuff inside later in replacement text. Note the \2 in replacements, fetching two times the first space amount.