Edit this page on our live server and create a PR by running command !create-pr in the console panel

Running cell in modified environments

  • Difficulty level: easy
  • Time need to lean: 10 minutes or less
  • Key points:
    • Magic %env --new runs the cell in a fresh SoS environment
    • Magic %env --tempdir runs the cell in a temporary directory that will be removed after the completion of the cell
    • Magic %env --expect-error expects an error from the cell
    • Magic %env --allow-error ignores errors from the cell
    • Magic %env --set KEY=VAR set environment variable
    • Magic %env --prepend-path PATH prepend PATH to $PATH

Running with fresh SoS environment

Sometimes when you are developing a script you would like to make sure that the script can be executed independently. In this case you can run it in a sandbox, which runs the cell in a fresh SoS dictionary.

For example, when you define a variable filename,

In [1]:

you can use it in other cells

In [2]:
     483 magic_env.ipynb

But you can not use it directly in a new SoS environment (thus cannot be used directly as a workflow step),

In [3]:
ExecuteError: [0]: 
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
script_522225025603812617 in <module>
      sh(fr"""wc -l {filename}
----> """)
      

NameError: name 'filename' is not defined

This would remind you that filename is not defined in the script and you will need to define it as a parameter and pass it from command line if you would like to expect it as a complete workflow with magic %run:

In [4]:
> %run --filename=magic_env.ipynb -v1
     483 magic_env.ipynb

Using a temporary directory (option --tempdir)

With option --tempdir, this magic will create a temporary directory and set it as the current directory before the execution of the cell, and remove the directory after the completion of the cell. For example, when you execute a workflow to create sandbox.txt with %env --tempdir, this file will not exist in the current directory after the completion of the cell.

In [5]:
In [6]:
ls: sandbox.txt: No such file or directory

Expect an error (option --expect-error)

Sometimes, for example, in the documentation of SoS, we would intentionally generate an error to demonstrate a problem. The error does not matter when we execute the notebook interactively, but would stop the execution of the notebook with Run All Cells from Jupyter, or execute the notebook in batch mode with papermill --engine sos.

To fix this problem, as what has been done in some previous cells of this document, you can use magic %env --expect-error, which let SoS know that the cell will return an error. The magic will return ok if an error is received, and actually error if an ok is received.

Allow errors (option --allow-error)

Similar to %env --expect-error, %env --allow-error will tolerate an error returned from the cell, so it will return ok if the cell returns error, or ok.

Set environment variables (option --set)

Option --set set environment variables for the execution of a cell, which is meant to be temporary since the variables will be unset as soon as the exection of the cell is completed.

The values of this option can be one or more of

  • KEY=VALUE pair which sets value of environment variable KEY to VALUE
  • KEY which sets value of environment variable KEY to ''
In [7]:
DEBUG
In [8]:




















Set PATH (option --prepend-path)














If you would like to use a command in a specific path instead of the one in system $PATH, you can use option env for actions (see SoS actions for details). You can also modifying the PATH of the current cell by using option --prepend-path.


For example,














In [9]:





















[1] "R version 3.6.1 (2019-07-05)"





















In [10]:





















[1] "R version 3.5.2 (2018-12-20)"