Regex to find and replace a specific character inside of multiple quoted strings in a string

Question

Given an example string of JSON (formatted here for reading, actual input may or may not be formatted):

{
   "page":1,
   "per_page":6,
   "total":12,
   "total_pages":14,
   "data":[
      {
         "id":1,
         "group:name":"Gray: cerulean",
         "year":2000,
         "color":"#98B2D1",
         "time" : "8:30",
         "pantone_value":"15-4020",
         "ratio" : "1:3:4"
      },
      {
         "id":2,
         "group:name":"Red: fuchsia rose",
         "year":2001,
         "color":"#C74375",
         "time" : "11:45",
         "pantone_value":"17-2031",
         "ratio" : "2:1:3"
      },
      {
         "id":3,
         "group:name":"Red: true red",
         "year":2002,
         "color":"#BF1932",
         "time" : "05:00",
         "pantone_value":"19-1664",
         "ratio" : "1:4:8"
      },
      {
         "id":4,
         "group:name":"Green: aqua sky",
         "year":2003,
         "color":"#7BC4C4",
         "time" : "09:15",
         "pantone_value":"14-4811",
         "ratio" : "3:4:5"
      },
      {
         "id":5,
         "group:name" : "Red: <no-name>",
         "year":2004,
         "color":"#E2583E",
         "time" : "16:50",
         "pantone_value":"17-1456",
         "ratio" : "2:0:1"
      },
      {
         "id":6,
         "group:name":"Blue: blue turquoise",
         "year":2005,
         "color":"#53B0AE",
         "time" : "22:05",
         "pantone_value":"15-5217",
         "ratio" : "5:1:2"
      }
   ]
}

I want to replace every colon character : INSIDE of a string value (i.e. bounded by a string dobule quotes) ONLY with some specific text (doesn't matter to the problem at hand). Ideally, I'd be searching/replacing all occurrences using regular expressions as a single shot operation, however I am not fully committed to that path if there is a more concise solution.

• The keys that are strings could contain colons, they could also very well not.
• The values are the same situation.
• There is no guarantee of schema, structure, order, etc.

As a starting point, I found a regular expression to capture all of the contents between quotes: "\"\\([^\"]*\\)\""

However, I have not found an elegant (or any) way to just capture any colon characters in the bounded string. If I could do that then it's just a simple matter of replacing the capture group. However, I cannot seem to get even just the strings with colons in them as the only thing captured.

Without resorting to naive solutions like iterating over eeach line and then capturing each string group in each line and doing the replacement twice (i.e. replace ":" in the string group and then the group in the line, for each group found in the line), is there a way to accomplish this?

Might it be worth digging into the json.el library or js-mode for utilities of some sort? Ideally, I'd like to avoid reaching deep into those libraries if possible because I am unsure of their stability overall and between Emacs versions.

Standard environment disclosure:

• OS: MacOS
• Chip: Intel
• Emacs Version: 27.2.1
• Port: Mitsuharu Yamamoto / Railwaycat
• Other Notes: - No special distributions / etc.
• Relevant Common Packages Used:
  • s.el
  • dash
  • all internal Emacs packages

Why am I doing this? I have some custom formatting code that keys off of colons to find the key-value separator, as per standard JSON. Unfortunately having colons inside of strings doesn't play nice and are not really discriminated against real colons being used as syntax separators.

Answer 1

Your regexp won't work for JSON strings containing escaped double-quotes like this:

"foo \":bar:\" baz"

You want a more robust regexp. Conveniently json-mode.el in GNU ELPA already provides C-hv json-mode-quoted-string-re:

(defconst json-mode-quoted-string-re
  (rx (group (char ?\")
             (zero-or-more (or (seq ?\\ ?\\)
                               (seq ?\\ ?\")
                               (seq ?\\ (not (any ?\" ?\\)))
                               (not (any ?\" ?\\))))
             (char ?\"))))

Which is rather more readable than:

"\\(\"\\(?:\\\\\\\\\\|\\\\\"\\|\\\\[^\"\\]\\|[^\"\\]\\)*\"\\)"

You can then process your text like so:

(defvar json-mode-quoted-string-re)
(defun my-replace-json-string-colons ()
  "Replace colons within JSON strings with a <COLON> token."
  (interactive)
  (require 'json-mode)
  (save-excursion
    (let ((bound (make-marker)))
      (while (re-search-forward json-mode-quoted-string-re nil :noerror)
        (set-marker bound (match-end 0))
        (goto-char (match-beginning 0))
        (while (search-forward ":" bound :noerror)
          (replace-match "<COLON>")))
      (set-marker bound nil))))

Answer 2

With Emacs it's generally better to operate on text in buffers, rather than in strings: easier and more efficient. Here, I'm assuming you'll really be doing that -- don't bother with strings at all.

Do you want to do this interactively or using code? Do you want to be able to check each occurrence of : in the contexts you describe, to say whether or not you want it replaced?

If you want to do it interactively then you can use Isearch+ -- to do that. You can define the zones (search contexts) you want to search or use query-replace(-regexp) over. Text outside those zones is ignored. (Same thing for non-query replacement commands.) You can define zones (to use as search/replace contexts) using regexps or any other way you like -- programmatically or interactively.