org-babel's matlab (session) output is malformed

Question

I'm trying to evaluate some Matlab code using org-babel in a session:

#+BEGIN_SRC matlab :session :results output
a = 1:10;
b = a + 3
c = a - 3
#+END_SRC

And following the org-babel documentation, this is the output I expect:

  b =

      4    5    6    7    8    9   10   11   12   13
  c =

    -2  -1   0   1   2   3   4   5   6   7

But this is the output I get:

  #+begin_example
  a = 1:10;
  b = a + 3

  b =

       4     5     6     7     8     9    10    11    12    13
  c = a - 3

  c =

      -2    -1     0     1     2     3     4     5     6     7
  'org_babel_eoe'

  ans =

      'org_babel_eoe'
  #+end_example

(Including the #+begin_example etc)

After going through the source of the function org-babel calls, it looks like here is what's supposed to happen:

1. org-babel-octave-evaluate-session copies the code into the Matlab REPL (calling matlab-shell if necessary) and the shell runs it.
2. It then copies the result back as a string up to the end-of-evaluation token, and then removes the original code from the output string.

it looks like org-babel-octave-evaluate-session is not reading the end-of-evaluation token ('org_babel_eoe') correctly, and not removing the input code from the resulting string either. This looks like it should be an easy fix, but I couldn't figure it out. The relevant function(s) are in /usr/share/emacs/27.0.90/lisp/org/ob-octave.el or equivalent location. I am using Emacs 27.0.90.

Any ideas how to fix this?

Notes:

1. The same org-babel module pulls double duty for Octave and Matlab with a flag to distinguish between uses, hence the name. When I run the same test as an Octave source block it runs correctly.
2. This issue regards evaluation of source blocks with the :results output option, where org-babel includes in the results block everything that the REPL prints to stdout. (This is different from the default :results value option where only the return value of the block is considered as a result.)

Answer 1

The function org-babel-octave-evaluate-session is bugged. Fixed it with a rewrite. It should work correctly on Matlab input now. If anyone else has trouble with Matlab and org-babel interfacing in session mode, copy the following to a file and run (require 'ob-octave-fix.el nil t):

;; Session evaluation of MATLAB in org-babel is broken, this goes some
;; way towards addressing the problem.
;;
;;- I replaced a `delq' with `delete', the `eq' test was failing on
;; blank strings
;;
;;- For results of type `output', concatenate all statements in the
;; block with appropriate separators (";", "," etc) and run one long
;; statment instead. Remove this statement from the raw result. This
;; produces much cleaner output.

(defun org-babel-octave-evaluate-session
    (session body result-type &optional matlabp)
  "Evaluate BODY in SESSION."
  (let* ((tmp-file (org-babel-temp-file (if matlabp "matlab-" "octave-")))
     (wait-file (org-babel-temp-file "matlab-emacs-link-wait-signal-"))
     (full-body
      (pcase result-type
        (`output
         (mapconcat
          #'org-babel-chomp
          (list (if matlabp
                        (multi-replace-regexp-in-string
                         '(("%.*$"                      . "")    ;Remove comments
                           (";\\s-*\n+"                 . "; ")  ;Concatenate lines
                           ("\\(\\.\\)\\{3\\}\\s-*\n+"  . " ")   ;Handle continuations
                           (",*\\s-*\n+"                . ", ")) ;Concatenate lines
                         body)
                      body)
                    org-babel-octave-eoe-indicator) "\n"))
        (`value
         (if (and matlabp org-babel-matlab-with-emacs-link)
         (concat
          (format org-babel-matlab-emacs-link-wrapper-method
              body
              (org-babel-process-file-name tmp-file 'noquote)
              (org-babel-process-file-name tmp-file 'noquote) wait-file) "\n")
           (mapconcat
        #'org-babel-chomp
        (list (format org-babel-octave-wrapper-method
                  body
                  (org-babel-process-file-name tmp-file 'noquote)
                  (org-babel-process-file-name tmp-file 'noquote))
              org-babel-octave-eoe-indicator) "\n")))))
     (raw (if (and matlabp org-babel-matlab-with-emacs-link)
          (save-window-excursion
            (with-temp-buffer
              (insert full-body)
              (write-region "" 'ignored wait-file nil nil nil 'excl)
              (matlab-shell-run-region (point-min) (point-max))
              (message "Waiting for Matlab Emacs Link")
              (while (file-exists-p wait-file) (sit-for 0.01))
              "")) ;; matlab-shell-run-region doesn't seem to
        ;; make *matlab* buffer contents easily
        ;; available, so :results output currently
        ;; won't work
        (org-babel-comint-with-output
            (session
             (if matlabp
             org-babel-octave-eoe-indicator
               org-babel-octave-eoe-output)
             t full-body)
          (insert full-body) (comint-send-input nil t)))) results)
    (pcase result-type
      (`value
       (org-babel-octave-import-elisp-from-file tmp-file))
      (`output
       (setq results
         (if matlabp
         (cdr (reverse (delete "" (mapcar #'org-strip-quotes
                          (mapcar #'org-trim (remove-car-upto-newline raw))))))
           (cdr (member org-babel-octave-eoe-output
                (reverse (mapcar #'org-strip-quotes
                         (mapcar #'org-trim raw)))))))
       (mapconcat #'identity (reverse results) "\n")))))

(defun remove-car-upto-newline (raw)
  "Truncate the first string in a list of strings `RAW' up to the first newline"
  (cons (mapconcat #'identity
                   (cdr (split-string-and-unquote (car raw) "\n"))
                   "\n") (cdr raw)))

(defun multi-replace-regexp-in-string (replacements-list string &optional rest)
  (interactive)
  "Replace multiple regexps in a string. Order matters."
  (if (null replacements-list)
      string
    (let ((regex (caar replacements-list))
          (replacement (cdar replacements-list)))
      (multi-replace-regexp-in-string (cdr replacements-list)
                                      (replace-regexp-in-string regex replacement
                                                                string rest)))))

(provide 'ob-octave-fix)

Available as a file here.

---

Explanation of the bug: There are two problems with this function:

1. The first is matlab-shell's behavior. When fed multi-line input it echoes all but the first line individually in the prompt. This is not the behavior of the Matlab IDE. The above function concatenates all lines into one long command (with appropriate separators) before feeding it to the REPL.

2. The end-of-eval token is not being removed from the output because of a check being made with eq instead of equal. I replaced a call to delq with a call to delete.

The output of matlab session code blocks is now as expected:

#+BEGIN_SRC matlab :session :results output
a = 1;
b = a + 3
c = a - 3
#+END_SRC

#+RESULTS:
: b =
:      4
: c =
:     -2