10

I have been using Emacs with ESS and tramp to load remote files and run remote R processes on HPC computers. With ssh keys, tramp makes this easy with C-x C-f to find file at /ssh:myserver:/path/to/file followed by M-x R to launch an R session.

However, on HPC computers that use a job scheduling system (e.g. Sun Grid Engine or Torque) there are restrictions on how much memory and CPU can be used on the head node, so I have to launch an interactive session using qsub -I.

Is it possible to configure tramp, ssh, and/or Emacs to launch an interactive session on a slave node (using qsub -I or equivalent) after an ssh connection is made?

  • I'm not entirely sure what the limitations are? – PythonNut Jun 4 '15 at 18:13
  • @PythonNut I can ssh to the head node using tramp, but do not know how to then connect to the slave node (which is not directly accessible via ssh). – David LeBauer Jun 4 '15 at 18:29
  • It does not seem to be possible to tell ESS to launch a different shell first, but with tramp-remote-shell set to qrsh or qlogin you might be able to get an interactive cluster session. – rekado Jun 4 '15 at 18:53
3

It sounds like you should extend tramp-methods, adding a new method which is similar to the sudo method but uses qsub -I instead. You would then be able to use a multi-hop tramp path to first connect to the head node and then to connect to a compute node.

Try this:

(add-to-list 'tramp-methods
  '("qsub"
    (tramp-login-program        "qsub")
    (tramp-login-args           (("-I"))) ; other options here?
    ;; Local $SHELL could be a nasty one, like zsh or fish.  Let's override it.
    (tramp-login-env            (("SHELL") ("/bin/sh")))
    (tramp-remote-shell         "/bin/sh")
    (tramp-remote-shell-args    ("-c"))
    (tramp-connection-timeout   10)))

You would then use a tramp path like /ssh:myserver|qsub:myserver:/path/to/file.

You may need to specify other options to pass to qsub; I've not used it so I can't help you there. You'll also want to look over the other options you can specify on a tramp method; there are a couple of dozen listed in tramp.el.

3

I have been struggling to do the same. I found a different route that allowed me to do basically the same thing.

You can open a shell buffer by M-x shell and from there connect to the login node, and then connect to the interactive session by qsub -I. Once you're in the interactive session, start an R session by typing the command R. There, you can do M-x ess-remote. This will prompt a mini-buffer asking which program you want to run (R, S+, Stata, etc.) Once you select R, then you can use the R session on the remote shell just as you use ESS in the local machine.

I personally open Rscripts saved in the remote node by tramping into it (in a separate window) and work with the R session connected by the ess-remote as described above. This way, the R session can directly interact with the Rscripts that I am working with. (For example, source('code.R') command in R will be able to read the 'code.R' in the HPC cluster, which I might have just edited. If I was editing Rscripts in my local machine instead, the ess-remote session of R wouldn't be able to read them unless I uploaded them to the remote node each time.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.