Edit this page on our live server and create a PR by running command !create-pr in the console panel

Setup and test remote hosts

  • Difficulty level: easy
  • Time need to lean: 10 minutes or less
  • Key points:
    • This command is experiemental and up to change without notice
    • sos remote list lists all defined hosts
    • sos remote setup sets up public-key authentication of specified host
    • sos remote test tests specified or all defined remote hosts
In [2]:
usage: sos remote [-h] [-c CONFIG] [-p PASSWORD] [--files [FILES ...]]
                  [--cmd ...] [-v {0,1,2,3,4}]
                  {list,status,setup,test,login,run} [hosts ...]

Listing and testing remote configurations

positional arguments:
  {list,status,setup,test,login,run}
                        List (list), check status of tasks (status), setup
                        public-key authentication (setup), test configuration
                        (test), login (login), push files to one or more
                        remote hosts (push), pull files from a remote host, or
                        execute command (run) on one or all or specified
                        remote hosts
  hosts                 Hosts to be checked or tested. All hosts defined in
                        SoS configurations will be included if unspecified. As
                        a special case for "sos remote setup", an address is
                        acceptable even if it is defined in configuration
                        file.

options:
  -h, --help            show this help message and exit
  -c CONFIG, --config CONFIG
                        A configuration file with host definitions, in case
                        the definitions are not defined in global sos
                        config.yml files.
  -p PASSWORD, --password PASSWORD
                        Password used to copy public key to remote hosts. You
                        will be prompted for a password if a password is
                        needed and is not passed from command line. The same
                        password will be used for all specified hosts so you
                        will need to use separate setup commands for hosts
                        with different passwords.
  --files [FILES ...]   files or directories to be push or pulled for action
                        "push" or "pull"
  --cmd ...             commands to be executed by action "run" or tested by
                        action "test". This option takes all remaining options
                        of part of command and must be the last option of the
                        sos remote command.
  -v {0,1,2,3,4}, --verbosity {0,1,2,3,4}
                        Output error (0), warning (1), info (2) and debug (3)
                        information to standard output (default to 2). More
                        debug information could be generated by setting
                        environmental variable SOS_DEBUG to comma separated
                        topics of GENERAL, WORKER, CONTROLLER, STEP, VARIABLE,
                        EXECUTOR, TARGET, ZERONQ, TASK, DAG, and ACTION, or
                        ALL for all debug information

Lists configured remote hosts (command sos remote list)

Command sos remote list lists all configured remote hosts

In [2]:
Alias     Address                    Queue Type Description                     
-----     -------                    ---------- -----------                     
bcb       bcbm-bpeng.mdanderson.edu  process                                    
htc       q1prphtch00.mdanderson.edu pbs        DQS HTC cluster (PBS)           
macpro    localhost                  process                                    
vm        192.168.47.140             process                                    
vm_rhel7  10.105.34.112              pbs        CentOS7 VM in office            
vm_tsp    192.168.47.140             pbs                                        
vm_ubuntu 10.192.91.75               pbs        Ubuntu VM in South Campus office

Test one or more host (command sos remote test)

In [3]:
Alias Address Queue Type ssh scp sos paths shared
----- ------- ---------- --- --- --- ----- ------
bcb   ?       ?          -   -   -   -     -

Set up public authentication (command sos remote setup host)

Command sos remote setup host takes your password from command line or prompt and tries to run commands such as

  • ssh-keygen -t rsa
  • scp ~/.ssh/id_rsa.pub host:
  • cat id_rsa.pub >> .ssh/authorized_keys on host
  • rm id_rsa.pub on host

It also tries to check the directory permissions. However, the command might fail from time to time due to differences in configurations of sshd service so you might still need to set up public-key authentication manually.