0

Given an example string of JSON (formatted here for reading, actual input may or may not be formatted):

{
   "page":1,
   "per_page":6,
   "total":12,
   "total_pages":14,
   "data":[
      {
         "id":1,
         "group:name":"Gray: cerulean",
         "year":2000,
         "color":"#98B2D1",
         "time" : "8:30",
         "pantone_value":"15-4020",
         "ratio" : "1:3:4"
      },
      {
         "id":2,
         "group:name":"Red: fuchsia rose",
         "year":2001,
         "color":"#C74375",
         "time" : "11:45",
         "pantone_value":"17-2031",
         "ratio" : "2:1:3"
      },
      {
         "id":3,
         "group:name":"Red: true red",
         "year":2002,
         "color":"#BF1932",
         "time" : "05:00",
         "pantone_value":"19-1664",
         "ratio" : "1:4:8"
      },
      {
         "id":4,
         "group:name":"Green: aqua sky",
         "year":2003,
         "color":"#7BC4C4",
         "time" : "09:15",
         "pantone_value":"14-4811",
         "ratio" : "3:4:5"
      },
      {
         "id":5,
         "group:name" : "Red: <no-name>",
         "year":2004,
         "color":"#E2583E",
         "time" : "16:50",
         "pantone_value":"17-1456",
         "ratio" : "2:0:1"
      },
      {
         "id":6,
         "group:name":"Blue: blue turquoise",
         "year":2005,
         "color":"#53B0AE",
         "time" : "22:05",
         "pantone_value":"15-5217",
         "ratio" : "5:1:2"
      }
   ]
}

I want to replace every colon character : INSIDE of a string value (i.e. bounded by a string dobule quotes) ONLY with some specific text (doesn't matter to the problem at hand). Ideally, I'd be searching/replacing all occurrences using regular expressions as a single shot operation, however I am not fully committed to that path if there is a more concise solution.

  • The keys that are strings could contain colons, they could also very well not.
  • The values are the same situation.
  • There is no guarantee of schema, structure, order, etc.

As a starting point, I found a regular expression to capture all of the contents between quotes: "\"\\([^\"]*\\)\""

However, I have not found an elegant (or any) way to just capture any colon characters in the bounded string. If I could do that then it's just a simple matter of replacing the capture group. However, I cannot seem to get even just the strings with colons in them as the only thing captured.

Without resorting to naive solutions like iterating over eeach line and then capturing each string group in each line and doing the replacement twice (i.e. replace ":" in the string group and then the group in the line, for each group found in the line), is there a way to accomplish this?

Might it be worth digging into the json.el library or js-mode for utilities of some sort? Ideally, I'd like to avoid reaching deep into those libraries if possible because I am unsure of their stability overall and between Emacs versions.

Standard environment disclosure:

  • OS: MacOS
  • Chip: Intel
  • Emacs Version: 27.2.1
  • Port: Mitsuharu Yamamoto / Railwaycat
  • Other Notes: - No special distributions / etc.
  • Relevant Common Packages Used:
    • s.el
    • dash
    • all internal Emacs packages

Why am I doing this? I have some custom formatting code that keys off of colons to find the key-value separator, as per standard JSON. Unfortunately having colons inside of strings doesn't play nice and are not really discriminated against real colons being used as syntax separators.

4
  • Looks like your custom formatting code is defective. Maybe, you should just use json-pretty-print instead. Select the full buffer as region and run M-x json-pretty-print.
    – Tobias
    Mar 23, 2023 at 11:18
  • You are probably better off using a JSON parser (there are probably dozens for any language you would wish: Python, Java, JS, maybe C, C++, C#, Rust etc) I don't know if there is one in Emacs Lisp though. E.g. in Python, you would do import json; jsdict = json.loads(open("/path/to/file.json").read()) and then you can use python functions to walk the dictionary and do whatever transformation to its elements you want, At the end, write it out with with open("/path/to/newfile.json") as f: f.write(json.dumps(jsdict)). Incomplete and untested of course, but you might ask on SO about it.
    – NickD
    Mar 23, 2023 at 13:31
  • 1
    BTW, the comma on l.45 in your example should probably be a colon. I tested that much :-)
    – NickD
    Mar 23, 2023 at 13:31
  • @NIckD BTW, json-pretty-print did detect the error with the comma. I changed it to a colon and the printer did its job (very well).
    – Tobias
    Mar 23, 2023 at 19:03

2 Answers 2

1

Your regexp won't work for JSON strings containing escaped double-quotes like this:

"foo \":bar:\" baz"

You want a more robust regexp. Conveniently json-mode.el in GNU ELPA already provides C-hv json-mode-quoted-string-re:

(defconst json-mode-quoted-string-re
  (rx (group (char ?\")
             (zero-or-more (or (seq ?\\ ?\\)
                               (seq ?\\ ?\")
                               (seq ?\\ (not (any ?\" ?\\)))
                               (not (any ?\" ?\\))))
             (char ?\"))))

Which is rather more readable than:

"\\(\"\\(?:\\\\\\\\\\|\\\\\"\\|\\\\[^\"\\]\\|[^\"\\]\\)*\"\\)"

You can then process your text like so:

(defvar json-mode-quoted-string-re)
(defun my-replace-json-string-colons ()
  "Replace colons within JSON strings with a <COLON> token."
  (interactive)
  (require 'json-mode)
  (save-excursion
    (let ((bound (make-marker)))
      (while (re-search-forward json-mode-quoted-string-re nil :noerror)
        (set-marker bound (match-end 0))
        (goto-char (match-beginning 0))
        (while (search-forward ":" bound :noerror)
          (replace-match "<COLON>")))
      (set-marker bound nil))))
1
  • I didn't know json-mode was even a thing. That definitely steered me in the right direction. Yeah, my regex was assuredly very rough and just a basic "got something working". Thanks. I tweaked a few things in my various functions (using a callback to generalize things) and ultimately got it accomplished using the regex you linked/provided. I wish Emacs had a fully modern day PCRE compatible regex implementation as I think it could be accomplished with look behind/ahead tokens ... maybe. Any who, thank you! :)
    – David
    Mar 23, 2023 at 17:14
1

With Emacs it's generally better to operate on text in buffers, rather than in strings: easier and more efficient. Here, I'm assuming you'll really be doing that -- don't bother with strings at all.

Do you want to do this interactively or using code? Do you want to be able to check each occurrence of : in the contexts you describe, to say whether or not you want it replaced?

If you want to do it interactively then you can use Isearch+ -- to do that. You can define the zones (search contexts) you want to search or use query-replace(-regexp) over. Text outside those zones is ignored. (Same thing for non-query replacement commands.) You can define zones (to use as search/replace contexts) using regexps or any other way you like -- programmatically or interactively.

2
  • Without getting into the weeds, it is a string and I'm just manipulating a copy of it and eventually do dump it into a temp buffer to do further manipulations. I wanted to avoid multiple instances of with-temp-buffer (though that may be more of an implementation detail). Any who, yes it would be progmmatically only. Though, I am curious, does query-replace... (QP) provide a more elegant way to do these kinds of replacements than the general approach used by phils (the while-loop with re-search)? I use this approach myself and found it very fast if verbose. Is QP faster and more concise?
    – David
    Mar 23, 2023 at 17:09
  • @David: Code is always more general than interactive: anything you can do interactively can be accomplished with code. (But you should consider using buffers rather than strings.)
    – Drew
    Mar 23, 2023 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.