Accessing next/previous parenthesis instead of just the outermost one? (as with `syntax-ppss`)

Question

syntax-ppss gives the surrounding parenthesis.

Is it possible to get the next/previous parenthesis using this API?
(which has the advantage of caching the syntax tree and working with multiple languages).

This would allow stepping over a region without having to start with the inner most nested part - working out.

Answer 1

It is more usual to search for sexps than for embraced expressions. Maybe that is the reason why you did not get a fast answer.

The usual way to search for stuff within program code avoiding comments and strings is to test (nth 8 (syntax-ppss)):

8. The string or comment start position. While inside a comment, this is the position where the comment began; while inside a string, this is the position where the string began. When outside of strings and comments, this element is nil.

You just continue the search if you end up in a comment or string. I demonstrate that for scanning forward for parentheses:

(while (progn
     (skip-syntax-forward "^()")
     (and (null (eobp))
          (nth 8 (syntax-ppss))))
  (forward-char))

The following convenience functions avoid strings and comments in both directions. These updated functions are a result of a fruitful discussion with npostavs.

(defun in-code-forward (fun &rest args)
  "Evaluate FUN with ARGS until we are out of comments and strings.
It is assumed that FUN does not move point backwards.  If point
ends up in a comment or string move point forward into code and
execute FUN again.  Stop execution when point is in code after
the execution of FUN or when the end of buffer is reached.
Return the result of the last evaluation of FUN."
  (let (ret state stop)
    (while
    (progn
      (setq ret (apply fun args)
        state (syntax-ppss)
        stop (point))
      (and (null (eobp))
           (nth 8 state)))
      (parse-partial-sexp stop
              (point-max)
              nil
              nil
              state
              'syntax-table))
    ret))

(defun in-code-backward (fun &rest args)
  "Evaluate FUN with ARGS until we are out of comments and strings.
It is assumed that FUN does not move point forwards.  If point
ends up in a comment or string move point backward into code and
execute FUN again.  Stop execution when point is in code after
the execution of FUN or when the beginning of buffer is reached.
Return the result of the last evaluation of FUN."
  (let (ret state stop)
    (while
    (progn
      (setq ret (apply fun args)
        state (syntax-ppss)
        stop (point))
      (and (null (bobp))
           (nth 8 state)))
      (goto-char (nth 8 state)))
    ret))

You can just write (in-code-forward #'skip-syntax-forward "^()") to search forward for parentheses and (in-code-backward #'skip-syntax-backward "^()") to search backward for parenthesis.
This is a very general approach. For an instance you can also search for a regexp in code with
(in-code-forward #'re-search-forward regexp nil 1).

Alternatively you can also extend some of the methods for searching forward for the next sexp.

The most basic function to jump to the next sexp is parse-partial-sexp. You can use that to search for sexps that start with a parenthesis:

(while (progn
     (parse-partial-sexp (point) (point-max) nil t (syntax-ppss))
     (null
      (or (eobp)
          (memq (car (syntax-after (point))) '(4 5)))))
  (forward-char))

This function assumes that you start your search outside of strings and comments.

Instead of parse-partial-sexp one also finds the combination (forward-sexp) (backward-sexp) in some libraries.

---

npostavs commented:

To get out of a comment it's much faster to do (goto-char (nth 8 (syntax-ppss)) for the backwards direction, or (parse-partial-sexp (point) (point-max) nil nil (syntax-ppss) 'syntax-table) for going forward.

If I interpret his comment right he recomments for the forward direction the following form:

(while (progn
       (skip-syntax-forward "^()")
       (and (null (eobp))
        (nth 8 (syntax-ppss))))
    (parse-partial-sexp (point) (point-max) nil nil (syntax-ppss) 'syntax-table))

Okay, let's measure it. I parse with the proposed forms the biggest elisp library simple.el.gz in the elisp base directory shipped with emacs 26.1 as representative example. As example application I count the syntactical parentheses in the simple.el.gz. There are also some comments with parentheses in that file.

(defun v1 ()
  "Skip to next parentheses using `forward-char' to escape comments."
  (while (progn
       (skip-syntax-forward "^()")
       (and (null (eobp))
        (nth 8 (syntax-ppss))))
    (forward-char)))

(defun v2 ()
  "Skip to next parentheses using `parse-partial-sexp' to escape comments."
  (while (progn
       (skip-syntax-forward "^()")
       (and (null (eobp))
        (nth 8 (syntax-ppss))))
    (parse-partial-sexp (point) (point-max) nil nil (syntax-ppss) 'syntax-table)))

(defun test-prog (skip-fun)
  "Profile SKIP-FUN counting parentheses in simple.el.gz."
  (with-temp-buffer
    (when (profiler-running-p)
      (profiler-stop))
    (find-library "simple.el.gz")
    (goto-char (point-min))
    (let ((cnt 0))
      (profiler-start 'cpu)
      (while (progn
           (funcall skip-fun)
           (null (eobp)))
    (cl-incf cnt)
    (forward-char))
      (prog1
      (profiler-report-cpu)
    (setq-local test-cnt cnt)
    (profiler-stop)))))


(with-current-buffer (test-prog #'v1)
  (rename-buffer "*Profile:v1*"))
(with-current-buffer (test-prog #'v2)
  (rename-buffer "*Profile:v2*"))

The output of the profile comparison is:

- v2                                          +775     
 - while                                      +771     
  - progn                                     +751     
   - and                                      +739     
    - nth                                     +731     
       syntax-ppss                            +715     
  - parse-partial-sexp                         +20     
     syntax-ppss                               +20     
- v1                                          -804     
 - while                                      -804     
  - progn                                     -800     
   - and                                      -792     
    - nth                                     -788     
       syntax-ppss                            -776  

So, npostavs is right that v2 is faster than v1, even if "much faster" is somewhat exaggerated. The gain makes about 4% for that minimal utilization of the scan (just incrementing a counter).

I am aware of the fact that this is not a comprehensive test.