Notebook environment for both interactive data analysis and batch data processing

Basics of SoS workflow system


More on SoS workflow system


Use SoS in SoS Notebook


The following tutorials are under development and we indicate the readiness of guides with (not ready), (in progress), and (ready). There are a lot to cover and there will be typos and errors here and there. If you spot an error or feel like contributing to the document, please

  1. Click the     button to the top right corner of each tutorial
  2. Execute the notebook and edit it on our live server
  3. Enter !create-pr <Shift-Enter> in the console panel to send us a PR


Please cite the following publication if you use SoS workflow engine for your work:

Script of Scripts: a pragmatic workflow system for daily computational research Gao Wang and Bo Peng (2019) PLoS Computational Biology. doi: 10.1371/journal.pcbi.1006843

Here are a list of examples from the publication:

Please follow instructions here to reproduce all but the last (due to data accessibility restrictions) examples.

Example Scripts and workflow notebooks

SoS scripts for the maintenance of SoS

  • update_toc.sos (script)
    A sos script that was used by SoS to update TOC of the wiki pages
  • release (script)
    Script to manage SoS website and release SoS to pip, docker hub, etc

Pedagogical examples

  • DEG Annotation with external task (notebook)
    The same example but submit the R part of the workflow as an external task
  • Simple External Tasks (notebook)
    An example to generate and execute a number of external tasks in Jupyter notebook. You can try to use option trunk_size to group the tasks. This example also demonstrates the use of option shared to collect results from tasks as variables
  • Remote Target (notebook)
    An example on the analysis of targets that reside on remote servers using target remote() and step option remote

Real-world applications