3

I'm using Emacs as a TCP client for a remote Python debugger I wrote. I can connect and interact with the server, but I'm not able to get the comint buffer used as the client to align input and output correctly.

To demonstrate, I execute my_debugger_test.py, pictured here for line numbers:

code with line numbers

The break point is hit, so I call my-debugger-client-start and a comint buffer opens, *my-debugger-client*.

call my-debugger-client-start

I enter next. This will tell the debugger to go to the next line. Instead, I get a syntax error telling me that the input was invalid. This is because whatever was sent to the debugger did not correspond to a do_* method and was not valid Python:

invalid input

I know this because if I put point at the start of one of the lines, I can call valid Python, such as print("hi") and the line will execute.

To demonstrate:

Move point to beginning of line

Go to front of line

Enter valid Python

insert print function

Press RET and the line evaluates

the line evaluates

We can see that "hi" was printed. Unfortunately, it was printed after the prompt. A NameError also shows up because the comint also sent (MyDebugger). If I run a separate Python instance, I can get the same result:

reproduce NameError

It seems that putting point at the start of the last line works and that the comint is seeing the prompt as input text. Again, go to the start of the line. This time, insert next to execute the current line:

enter next at the start of line

debugger moves to next line

We can see the y = 2 is printed showing us that the debugger executed do_next and moved to the next line.

Obviously, having the comint buffer behave like *shell* is desirable.

  • How can I get output to appear above the prompt?

  • How can I exclude the prompt when sending input to the debug server?

The server/debugger code is here. Type help in the client to see what commands it accepts. Currently, the prompt is defined in Python, the self.prompt = f'({self.__class__.__name__}) ' line. Maybe it needs to be defined in elisp?

# mydebugger.py

import pdb
import sys
import socket


class MyDebugger(pdb.Pdb):
    """

    Set up server/breakpoint in code:

        MyDebugger(host='0.0.0.0', port=4444).set_trace()

    Connect client:

        telnet 127.0.0.1 4444

    """

    def __init__(self, host, port, skip=None, complete_key='tab'):

        self.listen_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.listen_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
        self.listen_socket.bind((host, port))
        self.listen_socket.listen(1)

        connection, address = self.listen_socket.accept()

        self.stream = connection.makefile('rw', errors='ignore')

        self._old_stdin = sys.stdin
        self._old_stdout = sys.stdout

        sys.stdin = self.stream
        sys.stdout = self.stream

        pdb.Pdb.__init__(self, readrc=False)

        self.use_rawinput = False

        self.prompt = f'({self.__class__.__name__}) '

    def _clean_up(self):
        self.stream.close()
        self.listen_socket.shutdown(socket.SHUT_RDWR)
        self.listen_socket.close()

        sys.stdin = self._old_stdin
        sys.stdout = self._old_stdout

    def do_quit(self, arg):
        self._clean_up()
        super().do_quit()

    def do_EOF(self, arg):
        self._clean_up()
        super().do_EOF()


def breakpoint():
    MyDebugger(host='127.0.0.1', port=4444).set_trace(sys._getframe().f_back)

The test code is:

# my_debugger_test.py

def first(a, b):
    print(f"entered first", flush=True)
    return a + b


def main():
    print(f"entered main", flush=True)
    import mydebugger; mydebugger.breakpoint()
    x = 1
    y = 2
    return first(x, y)


if __name__ == '__main__':
    value = main()
    print(f"value: {value}", flush=True)

Put mydebugger.py in the site-packages directory and run my_debugger_test.py. It will break and start a debug server you can connect to at 127.0.0.1:4444.

My Emacs client is modified from this minimal TCP client. Instead of printing to a buffer, it sends a message to the minibuffer. It also creates a *listen* buffer. If comint-mode is turned on there, you can send input to the server.

I've modified the minimal client to write to the buffer associated with the process:

(defvar my-debugger-client-port 4444
    "port of the server")

(defvar my-debugger-client-host "127.0.0.1"
    "host of the server")

(defun my-debugger-client-start nil
  "starts an emacs tcp client my-debugger-clienter"
  (interactive)
  (make-network-process
   :name "my-debugger-client"
   :buffer "*my-debugger-client*"
   :family 'ipv4
   :host my-debugger-client-host
   :service my-debugger-client-port
   :sentinel 'my-debugger-client-sentinel
   :filter 'my-debugger-client-filter)

  (switch-to-buffer "*my-debugger-client*")

  (with-current-buffer "*my-debugger-client*"
    (comint-mode)))

(defun my-debugger-client-stop nil
  "stop an emacs tcp my-debugger-clienter"
  (interactive)
  (delete-process "my-debugger-client"))

(defun my-debugger-client-filter (proc string)
  (with-current-buffer "*my-debugger-client*"
    (insert string)))

(defun my-debugger-client-sentinel (proc msg)
  (when (string= msg "connection broken by remote peer\n")
    (with-current-buffer "*my-debugger-client*"
      (insert (format "client %s has quit" proc)))))
3
  • 1
    Perhaps add an annotated screenshot to show the problem for people who might be able to help but aren't going to actually run and test your python code.
    – phils
    Jan 25, 2022 at 6:47
  • This smells like a prompt problem to me: see e.g. the contortions that Org babel sessions have to go through to deal with the prompt in the inferior process. But I haven't looked at your question in any detail, so I could be off by miles, if not by parsecs.
    – NickD
    Jan 25, 2022 at 20:36
  • @NickD, that sounds reasonable. I had looked at shell and run-python for how they handle prompts. I had not thought to look at babel sessions. Thanks for the suggestion. Jan 25, 2022 at 21:05

1 Answer 1

1

The buffer created by make-network-process, which was named *my-debugger-client* is what's called a process buffer. These are described in the Processes section of the manual.

The manual says,

The position to insert the output is determined by the ‘process-mark’, which is then updated to point to the end of the text just inserted. Usually, but not always, the ‘process-mark’ is at the end of the buffer.

Regarding input, the RET in a process buffer defaults to command comint-send-input. The help for this command says,

After the process output mark, [comint-send-input] sends all text from the process mark to point as input to the process.

So, to answer the questions "How can I get output to appear above the prompt?" and "How can I exclude the prompt when sending input to the debug server?", you need to pay attention to where the process mark and point are.

As pointed out in the question, the prompt affects the calculation of input and output in regards to the process mark and point. Searching the comint.el source, there are several variables dealing with prompts, comint-prompt-regexp and comint-use-prompt-regexp. The first defines the prompt, the second tells Emacs to use the regexp to distinguish input and output regions.

Finally, there's the bookkeeping involved with using the prompt regexp to place text in the process buffer such that output is above the prompt and input is read from after it. That's exactly what the comint-output-filter function does.

So, all together, set the comint-prompt-regexp and comint-use-prompt-regexp in the process buffer. Also, replace the my-debugger-client-filter to wrap comint-output-filter.

(defvar my-debugger-client-port 4444
    "port of the server")

(defvar my-debugger-client-host "127.0.0.1"
    "host of the server")

(defun my-debugger-client-start nil
  "starts an emacs tcp client my-debugger-clienter"
  (interactive)
  (make-network-process
   :name "my-debugger-client"
   :buffer "*my-debugger-client*"
   :family 'ipv4
   :host my-debugger-client-host
   :service my-debugger-client-port
   :sentinel 'my-debugger-client-sentinel
   :filter 'my-debugger-client-filter)

  (switch-to-buffer "*my-debugger-client*")

  (with-current-buffer "*my-debugger-client*"
    (comint-mode)
    (setq-local comint-prompt-regexp "(MyDebugger) ")
    (setq-local comint-use-prompt-regexp t)))

(defun my-debugger-client-stop nil
  (interactive)
  (delete-process "my-debugger-client"))

(defun my-debugger-client-filter (proc string)
  (comint-output-filter proc string))

(defun my-debugger-client-sentinel (proc msg)
  (when (string= msg "connection broken by remote peer\n")
    (with-current-buffer "*my-debugger-client*"
      (insert (format "client %s has quit" proc)))))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.