SoS (Script of Scripts) is the umbrella title for SoS Polyglot Notebook and SoS Workflow Engine. SoS Notebook is a customized Jupyter kernel with associated frontend extensions that allows the use of multiple kernels in one Jupyter notebook. SoS Workflow engine is a Python 3 based workflow engine designed to be readable, sharable, and suitable for daily data analysis.
|SoS Notebook||SoS Workflow System|
SoS Notebook and SoS workflow engine share the same website because they are designed to work with each other to facilitate both interactive data analysis and batch data processing.
NOTE: Notebooks for these three animations are available under directory examples in our live SoS server.
As an interactive environment and notebook tool that promotes literate programming, SoS allows you to perform and record your analysis in different languages in a single Jupyter notebook, with seamless integration of multiple Jupyter kernels (e.g. python, and R). The ability to exchange data between live jupyter kernels allows you to use the most appropriate languages for different parts of your analyses.
SoS can work with all Jupyter kernels and currently provides native support (with data exchange) for the following languages:
SoS follows your definition of workflow that allows you to define workflows in both foward and makefile style, even a mixture of both. More specifically, you can define workflows as ordered steps that would be executed sequentially step by step to process specified input files; or as a set of makefile-style rules that would be executed to generated specified output files; or as a sequence of steps with help from makefile-style steps. SoS automatically analyzes the input and output of steps and execute workflows using a dynamic DAG (Directed Acyclic Graph) that would be expanded and shrinked during the execution.
SoS is very easy to learn so that you can start using it in minutes without having to learn any of the advanced features. As demonstrated in our Quick Start Guide, you can easily consolidate your scripts into an executable workflow using SoS and add more features to the script step by step if necessary. The multi-kernel feature of the Jupyter notebook is also easy to use with only two magics to remember.