Creating a Remote Plan when R is in an Environment Module

[lpiep]

I have access to a Sun Grid Engine cluster, and I've successfully set up future.batchtools to run from the head node on that machine. What I'd now like to do is implement a nested future that I can run from my local server. As a first step, I'm trying to set up a remote future that executes code on the head node of that remote server from my local server.

The trouble I'm running into is that R is activated via an environment module on the remote machine. That is, if I were logging in to run R as normal, I'd do:

ssh remote.server.com
module load R
R 

My understanding is that future requires Rscript to be available on the worker (in this case remote.server.com). Is there some way to tell it that it needs to run module load R before attempting to use Rscript?

[HenrikBengtsson]

Hello. I think the easiest approach is to create a custom shell script that replaces Rscript when it comes to launching the R worker. Assuming you use Bash as your shell script on the remote system, create a myRscript script like:

#! /usr/bin/env bash

## Load the 'r' module
module load r

## Call Rscript and pass all arguments as-is
Rscript "$@"

Make sure it's executable, i.e. do:

{remote}$ chmod ugo+x myRscript

Verify it works;

{remote}$ ~/myScript --version
Rscript (R) version 4.2.1 (2022-06-23)

{remote}$ ~/myRscript -e "getRversion()"
[1] ‘4.2.1’

and also from your local computer, e.g.

{laptop}$ ssh remote.server.com '~/myRscript -e "getRversion()"'
[1] ‘4.2.1’

When you've verified all of the above, then you should be able to launch a parallel worker using the following "low-level" parallelly code:

{laptop}$ R
...
> cl <- parallelly::makeClusterPSOCK("remote.server.com", rscript = "myRscript")
> print(cl)
Socket cluster with 1 nodes where 1 node is on host 'remote.server.com' (R version 4.2.1 (2022-06-23), platform x86_64-pc-linux-gnu)

When you've confirmed that working, you should be able to do:

{laptop}$ R
...
> library(future)
> cl <- parallelly::makeClusterPSOCK("remote.server.com", rscript = "myRscript")
> plan(cluster, workers = cl)
> f <- future(Sys.info()[["nodename"]])
> value(f)
[1] "remote.server.org"

or shorter,

> library(future)
> plan(cluster, workers = "remote.server.com", rscript = "myRscript")
> f <- future(Sys.info()[["nodename"]])
> value(f)
[1] "remote.server.org"

Hope this helps

[HenrikBengtsson]

• I needed to add myRscript to my ~/bin and escape the parens to get the remote command working (ssh remote.server.com myRscript -e "getRversion\(\)")

Correction on my end; yes, you're right, you have to escape like that, or use and layer of quotation marks when you call it over SSH. Here's a modified version (also corrected above):

{laptop}$ ssh remote.server.com '~/myRscript -e "getRversion()"'
[1] ‘4.2.1’

[lpiep]

Thanks again, Henrik.

The single quotes did work for me, but oddly still getting an issue when I use Rscript "$@":

{laptop}$ ssh remote.server.com 'Rscript-4.1.0 -e "getRversion()"'
Fatal error: cannot open file '-e getRversion()': No such file or directory

WRT my bash versions, both commands you asked about give GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu). That appears to be several years old (out of my control, I'm afraid), so perhaps that's the issue.

[HenrikBengtsson]

The single quotes did work for me, but oddly still getting an issue when I use Rscript "$@":

{laptop}$ ssh remote.server.com 'Rscript-4.1.0 -e "getRversion()"'
Fatal error: cannot open file '-e getRversion()': No such file or directory

There must be something odd in you're script. Can you please try:

{remote}$ Rscript-4.1.0 -e "getRversion()"

while logged in on that remote system? Let's worry about SSH later.

For troubleshooting purposes, add a few debug statements to your Rscript-4.1.0;

#! /usr/bin/env bash

## Load the 'r' module
module load r

## Debugging info
Rscript --version
echo "Command-line call: '$0 $*'"
echo "Command-line arguments [n=${#@}]: $@"

## Call Rscript and pass all arguments as-is
Rscript "$@"

WRT my bash versions, both commands you asked about give GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu). That appears to be several years old (out of my control, I'm afraid), so perhaps that's the issue.

Bash v4 should definitely be fine. I've verified that it works with the exact same Bash version on our CentOS 7.9 systems (which looks like you're using too).

[lpiep]

Ah, I realized I left out a detail when trying to simplify my explanation. My HPC environment also requires me to set a group with newgrp when I log in. Per this discussion on SO, I used a herefile in my script in order to get around the fact that newgrp starts a subshell.

So, my Rscript file on the remote machine is :

#! /usr/bin/env bash

## Load the 'R' module
newgrp mygroup << EOF
module load R/R-4.1.0

## Call Rscript and pass all arguments as-is
Rscript $(printf " %q" "${@}") # <-- this is where "$@" fails
EOF

That said, this script is currently working for me and I'm able to submit jobs to SGE on the remote server using my local R instance!

[HenrikBengtsson]

I see. Yeah, I can see how using a shell Here Document ("heredoc") complicates the quoting that otherwise is taken care of by "$@". That is beyond my Bash knowledge. My concern would be that there's some corner cases that $(printf " %q" "${@}") does not support. It would be great if there's some official Bash documentation saying this is the way to do it.

Having said that, I'm happy to hear it working for you now.