Edit this page on our live server and create a PR by running command !create-pr in the console panel

Previewing variables and files

  • Difficulty level: easy
  • Time need to lean: 10 minutes or less
  • Key points:
    • Magic %preview previews variables and files in supported format
    • Output of magic %preview can be saved in notebook or in console panel

Instant preview of intermediate results is extrmely useful for interactive data analysis. SoS provides rich and extensible preview features to

  • preview variables in SoS and subkernels,
  • preview files in many different formats,
  • show preview results temporarily in the side panel or permanently in the main notebook
  • generate interactive tables and plots for better presentation of data both in Jupyter notebook and in converted HTML reports

Magic %preview

SoS provides a %preview magic to preview files and variables after the completion of a cell. Although most of the time the item specified is the name of either a file or a variable, the magic will actually show information about both a file and a variable if they happen to have the same name, as shown in the following example.

In [1]:
> df (10 B):
1 line
something
> df: DataFrame of shape (3, 2)
  A   B  
0 A 1
1 B 2
2 C 3

%preview options

In [2]:
usage: %preview [-h] [-k KERNEL] [-w] [-s {table,scatterplot,png}] [-r HOST]
                [-p | -n] [-c CONFIG]
                [items [items ...]]

Preview files, sos variables, or expressions in the side panel, or notebook if
side panel is not opened, unless options --panel or --notebook is specified.

positional arguments:
  items                 Filename, variable name, or expression. Wildcard
                        characters such as '*' and '?' are allowed for
                        filenames.

optional arguments:
  -h, --help            show this help message and exit
  -k KERNEL, --kernel KERNEL
                        kernel in which variables will be previewed. By
                        default the variable will be previewed in the current
                        kernel of the cell.
  -w, --workflow        Preview notebook workflow
  -s {table,scatterplot,png}, --style {table,scatterplot,png}
                        Option for preview file or variable, which by default
                        is "table" for Pandas DataFrame. The %preview magic
                        also accepts arbitrary additional keyword arguments,
                        which would be interpreted by individual style.
                        Passing '-h' with '--style' would display the usage
                        information of particular style.
  -r HOST, --host HOST  Preview files on specified remote host, which should
                        be one of the hosts defined in sos configuration
                        files.
  -p, --panel           Preview in side panel even if the panel is currently
                        closed
  -n, --notebook        Preview in the main notebook.
  -c CONFIG, --config CONFIG
                        A configuration file with host definitions, in case
                        the definitions are not defined in global or local sos
                        config.yml files.

Preview in notebook or panel (options -n and -p)

By default, %preview displays results in the side panel if the side panel is open, and otherwise in the main notebook. You can override this behavior with options -p (--panel) or -n (--notebook) to always display results in the console panel or notebook. This is why we use -n most of the time in this tutorial to keep the preview results in this notebook.

For example, with option -p, the preview result will not be saved in notebook, even if the console panel is not open.

In [3]:

Preview embedded SoS workflow (option --workflow)

If option -w (--workflow) is specified, sos collects workflow steps from the current notebook and preview it. This would give you a better sense of what would be converted with magic %convert --to sos or executed with magic %sosrun.

In [4]:
In [5]:
In [6]:
#!/usr/bin/env sos-runner
#fileformat=SOS1.0

[step_1]
print('Step 1 of workflow')

[step_2]
print('Step 2 of workflow')

Preview variables from another kernel (option -k, --kernel)

The %preview magic previews the variables with the kernel using which the cell is evaluated. For example, even if you have var defined in both SoS and R kernel,

In [7]:
In [8]:
> val: list of length 4
[2, 3, 2, 3]
In [9]:
In [10]:
> val: 
 num [1:6] 1 2 1 2 1 2

If you would like to preview variable in another kernel, you can specify it using option --kernel, for example, the following cell previews a variable r.val in a SoS kernel.

In [11]:
> val: 
 num [1:6] 1 2 1 2 1 2

Preview files on remote host (option -r)

When a workflow or a task is executed remotely (see Remote Execution for details, result files might be on a remote host that is unavailable for local preview.

For example, you can execute a workflow on a remote host bcb to generate a file (mygraphic.png) and use %preview -r bcb to preview it.

In [12]:
INFO: No matching tasks are identified. Use option -a to check all tasks.
INFO: Running default: 
null device 
          1 
INFO: default is completed.
INFO: Workflow default (ID=18c4690c87e02c14) is executed successfully with 1 completed step.
> mygraphic.png (13.1 KiB):
No description has been provided for this image

Preview of variables

Preview variable in SoS kernel

In [13]:
> var1: list of length 3
['apple', 'banana', 'cherry']
> var2: dict of length 3
{'apple': 5, 'banana': 6, 'cherry': 6}

Preview variables in subkernels

In [14]:
> mtcars: 
'data.frame':	32 obs. of  11 variables:
 $ mpg : num  21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
 $ cyl : num  6 6 4 6 8 6 8 4 4 6 ...
 $ disp: num  160 160 108 258 360 ...
 $ hp  : num  110 110 93 110 175 105 245 62 95 123 ...
 $ drat: num  3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
 $ wt  : num  2.62 2.88 2.32 3.21 3.44 ...
 $ qsec: num  16.5 17 18.6 19.4 17 ...
 $ vs  : num  0 0 1 1 0 1 0 1 1 1 ...
 $ am  : num  1 1 1 0 0 0 0 0 0 0 ...
 $ gear: num  4 4 4 3 3 3 3 4 4 4 ...
 $ carb: num  4 4 1 1 2 1 4 2 2 4 ...
In [15]:

Preview of pandas DataFrame

If a data to be previewed is a Pandas DataFrame (or a csv or excel file which would be previewed as a DataFrame), SoS will preview it as a sortable and searchable table. For example, the following cell get a data.frame mtcar from the R kernel (as a pandas DataFrame) and preview it in the main notebook:

In [16]:
> mtcars: DataFrame of shape (32, 11)
  mpg   cyl   disp   hp   drat   wt   qsec   vs   am   gear   carb  
Mazda RX4 21.0 6.0 160.0 110.0 3.90 2.620 16.46 0.0 1.0 4.0 4.0
Mazda RX4 Wag 21.0 6.0 160.0 110.0 3.90 2.875 17.02 0.0 1.0 4.0 4.0
Datsun 710 22.8 4.0 108.0 93.0 3.85 2.320 18.61 1.0 1.0 4.0 1.0
Hornet 4 Drive 21.4 6.0 258.0 110.0 3.08 3.215 19.44 1.0 0.0 3.0 1.0
Hornet Sportabout 18.7 8.0 360.0 175.0 3.15 3.440 17.02 0.0 0.0 3.0 2.0
Valiant 18.1 6.0 225.0 105.0 2.76 3.460 20.22 1.0 0.0 3.0 1.0
Duster 360 14.3 8.0 360.0 245.0 3.21 3.570 15.84 0.0 0.0 3.0 4.0
Merc 240D 24.4 4.0 146.7 62.0 3.69 3.190 20.00 1.0 0.0 4.0 2.0
Merc 230 22.8 4.0 140.8 95.0 3.92 3.150 22.90 1.0 0.0 4.0 2.0
Merc 280 19.2 6.0 167.6 123.0 3.92 3.440 18.30 1.0 0.0 4.0 4.0
Merc 280C 17.8 6.0 167.6 123.0 3.92 3.440 18.90 1.0 0.0 4.0 4.0
Merc 450SE 16.4 8.0 275.8 180.0 3.07 4.070 17.40 0.0 0.0 3.0 3.0
Merc 450SL 17.3 8.0 275.8 180.0 3.07 3.730 17.60 0.0 0.0 3.0 3.0
Merc 450SLC 15.2 8.0 275.8 180.0 3.07 3.780 18.00 0.0 0.0 3.0 3.0
Cadillac Fleetwood 10.4 8.0 472.0 205.0 2.93 5.250 17.98 0.0 0.0 3.0 4.0
Lincoln Continental 10.4 8.0 460.0 215.0 3.00 5.424 17.82 0.0 0.0 3.0 4.0
Chrysler Imperial 14.7 8.0 440.0 230.0 3.23 5.345 17.42 0.0 0.0 3.0 4.0
Fiat 128 32.4 4.0 78.7 66.0 4.08 2.200 19.47 1.0 1.0 4.0 1.0
Honda Civic 30.4 4.0 75.7 52.0 4.93 1.615 18.52 1.0 1.0 4.0 2.0
Toyota Corolla 33.9 4.0 71.1 65.0 4.22 1.835 19.90 1.0 1.0 4.0 1.0
Toyota Corona 21.5 4.0 120.1 97.0 3.70 2.465 20.01 1.0 0.0 3.0 1.0
Dodge Challenger 15.5 8.0 318.0 150.0 2.76 3.520 16.87 0.0 0.0 3.0 2.0
AMC Javelin 15.2 8.0 304.0 150.0 3.15 3.435 17.30 0.0 0.0 3.0 2.0
Camaro Z28 13.3 8.0 350.0 245.0 3.73 3.840 15.41 0.0 0.0 3.0 4.0
Pontiac Firebird 19.2 8.0 400.0 175.0 3.08 3.845 17.05 0.0 0.0 3.0 2.0
Fiat X1-9 27.3 4.0 79.0 66.0 4.08 1.935 18.90 1.0 1.0 4.0 1.0
Porsche 914-2 26.0 4.0 120.3 91.0 4.43 2.140 16.70 0.0 1.0 5.0 2.0
Lotus Europa 30.4 4.0 95.1 113.0 3.77 1.513 16.90 1.0 1.0 5.0 2.0
Ford Pantera L 15.8 8.0 351.0 264.0 4.22 3.170 14.50 0.0 1.0 5.0 4.0
Ferrari Dino 19.7 6.0 145.0 175.0 3.62 2.770 15.50 0.0 1.0 5.0 6.0
Maserati Bora 15.0 8.0 301.0 335.0 3.54 3.570 14.60 0.0 1.0 5.0 8.0
Volvo 142E 21.4 4.0 121.0 109.0 4.11 2.780 18.60 1.0 1.0 4.0 2.0

Compared to previewing the same variable in R, you have the addition features of

  1. sorting table by clicking the sort icon at the header of each column
  2. displaying a subset of rows that matches specified text in the input text

Note that SoS by default outputs the first 200 rows of the table. You can use option -l (--limit) to change this threshold.

Preview of files

The %preview magic can be used to preview files in a variety of formats.

Preview of figures

SoS can preview figures in multiple formats including .png, .jpg, and .gif. For example, the following magic previews a png figure generated in R.

In [17]:
pdf: 2
> test.png (20.5 KiB):
No description has been provided for this image

Preview of pdf files

Note that SoS by default preview PDF files as an embeded object (iframe), but you can use option -s png to convert PDF to png to preview it as a image, if you have imagematick and Python package wand installed.

Option description
--style png (-s png) Convert PDF to png before preview. All pages are combined to produce a sngle PNG figure if the pdf file contains multiple pages. This option requires Python module wand.
--pages 2 3 With --style png, preview specified pages (page numbers starting from 1) from a multi-page PDF file.

Preview of bioinformatics-specific formats (module sos-bioinfo)

SoS uses a modular design and allows the definition of previewers in separate modules. For example, with the installation of module sos-bioinfo (and required packages such as pysam), you can preview bioinformatics-specific formats such as .VCF and .bam.

In [18]:
Out[18]:
0
> issue225.bam (4.2 KiB):
RG:
  ID: '16-3584'    PL: 'illumina'    PU: '16-3584'    SM: '16-3584'    
SQ:
  SN: 'chrM'    LN: 16571