Escape all square brackets with replace-regexp-in-string

Question

So I was writing a function that would convert one type of a link (e.g. Org-mode link) to another type (e.g. Markdown), and then I realized that Org-mode doesn't require you to escape square brackets, i.e.,

if you have a link like this:

[[https://github.com/foo/bar/issues/42][Major issue in [Bar] project]]

Org mode would have no issues with displaying it (whereas Markdown would). And if you try to convert it to Markdown, you have to escape those square brackets.

Of course, I could do something like the following:

(let ((str "Major issue in [Bar] project"))
  (replace-regexp-in-string "\\]" "\\\\]"
   (replace-regexp-in-string "\\[" "\\\\[" str))) 
   
;; => "Major issue in \\[Bar\\] project"

But I think it's possible to achieve the same result by calling the function only once and using optional arguments. Can someone show me how does it work?

Answer 1

I have figured it out. It looks like the second argument can be either regexp or a function. So this worked for me:

(replace-regexp-in-string
 "\\[\\|\\]"
 (lambda (x) (format "\\\\%s" x))
 "Major issue in [Bar] project")

;; => "Major issue in \\[Bar\\] project"

But ultimately, I still don't understand how other parameters, like I have figured it out. It looks like the second argument can be either regexp or a function. So this worked for me:

(replace-regexp-in-string
 "\\[\\|\\]"
 (lambda (x) (format "\\\\%s" x))
 "Major issue in [Bar] project")

;; => "Major issue in \\[Bar\\] project"

But ultimately, I still don't understand how other parameters, like SUBEXP work with this function

Update: I popped the question in Doom Emacs Discord and got help from amazing Henrik Lissner himself. Here are the examples he provided, they do help understanding the mechanics better:

(replace-regexp-in-string "^\\(foo\\)bar$" "bar\\1" "foobar")
;; #=> "barfoo"

(replace-regexp-in-string "^\\(foo\\)bar$" "baz\\&" "foobar")
;; #=> "bazfoobar"

(replace-regexp-in-string "^\\(foo\\)bar$" "bar\\1\\&" "foobar" nil t) ; literal = t
;; #=> "bar\\1\\&"

(replace-regexp-in-string "^\\(foo\\)bar$" "bar\\\\1\\\\&" "foobar") ; equivalent to literal = t
;; #=> "bar\\1\\&"
``` work with this function