Extending SoS

Difficulty level: intermediate
Time need to lean: 20 minutes or less
Key points:
- SoS uses entry points to register actions, targets etc, which allows third-party to define such components.

SoS can be easily extended with new actions, targets, converters, file previewers. To make the extension available to other users, you can either create and distribute a separate package, or extend SoS and send us a pull request. Please open a ticket and discuss the idea with us before you send a pull request.

Understanding `entry_points`

SoS makes extensive use of entry points, which allows external modules to register their features in the file system to make them available to other modules. It can be confusing initially but this stack overflow ticket explains the entry_points mechanism quite well.

To register additional feature with SoS, your package should define one or more sos-recognizable entry_points such as sos-languages, sos-targets, and sos-actions, with a syntax similar to

entry_points='''
[sos-language]
ruby = sos_ruby.kernel:sos_ruby

[sos-targets]
Ruby_Library = sos_ruby.target:Ruby-Library
'''

With the installation of this package, sos would be able to obtain a class sos_ruby from module sos_ruby.kernel, and use it to work with the ruby language.

Defining your own actions

Under the hood an action is a normal Python function that is decorated as SoS_Action. The decorator defines the common interface of actions and calls the actual function. To define your own action, you generally need to

from sos.actions import SoS_Action

@SoS_Action()
def my_action(*args, **kwargs):
    pass

The decorator accepts an optional parameter acceptable_args=['*'] which can be used to specify a list of acceptable parameter (* matches all keyword args). An exception will be raised if an action is defined with a list of acceptable_args and is called with an unrecognized argument.

You then need to add an entry to entry_points in your setup.py file as

[sos-actions]
my_action = mypackage.mymodule:my_action

The most important feature of an SoS actions is that they can behave differently in different run_mode, which can be dryrun, run, or interactive (for SoS Notebook). Depending on the nature of your action, you might want to do nothing for in dryrun mode and give more visual feedback in interactive mode. The relevant code would usually look like

if env.config['run_mode'] == 'dryrun':
    return None

Because actions are often used in script format with ignored return value, actions usually return None for success, and raise an exception when error happens.

If the execution of action depends on some other targets, you can raise an UnknownTarget with the target so that the target can be obtained, and the SoS step and the action will be re-executed after the target is obtained. For example, if your action depends on a particular R_library, you can test the existence of the target as follows:

from sos.targets import UnknownTarget
from sos.targets_r import R_library

@SoS_Action()
def my_action(script, *args, **kwargs):
    if not R_library('somelib').target_exists():
        raise UnknownTarget(R_library('somelib'))
    # ...

Additional targets

Additional target should be derived from BaseTarget.

from sos.targets import BaseTarget

class my_target(BaseTarget):
    def __init__(self, *args, **kwargs):
        super(my_target, self).__init__(self)
        
    def target_name(self):
        ...
    
    def target_exists(self, mode='any'):
        ...
       
    def target_signature(self):
        ...

Any target type should define the three functions:

target_name: name of the target for reporting purpose.
target_exists: check if the target exists. This function accepts a parameter mode which can target, signature, or any, which you can safely ignore.
target_signature: returns any immutable Python object (usually a string) that uniquely identifies the target so that two targets can be considered the same (different) if their signatures are the same (different). The signature is used to detect if a target has been changed.

The details of this class can be found at the source code of BaseTarget. The R_Library provides a good example of a virtual target that does not have a fixed corresponding file, can be checked for existence, and actually attempts to obtain (install a R library) the target when it is checked.

After you defined your target, you will need to add an appropriate entry point to make it available to SoS:

[sos-targets]
my_target = mypackage.targets:my_target

File format conversion

To convert between sos and another file format, you would need to define a class, with member functions get_parser() and convert().

Suppose you would like to convert .sos to a .xp format, you can define these the class as follows

import argparse
from sos.parser import SoS_Script

class ScriptToXpConverter():

    def get_parser(self):
        parser = argparse.ArgumentParser(
            'sos convert FILE.sos FILE.xp (or --to xp)',
            description='''Convert a sos script to XP (.xp).''')
        return parser

    def convert(self, script_file, xp_file, args=None, unknown_args=None):
        pass

You can then register the converter in setup.py as

[sos-converters]
sos-xp: mypackage.mymodule:ScriptToXpConverter

Here fromExt is file extension without leading dot, toExt is destination file extension without leading dot, or a format specified by the --to parameter of command sos convert. If dest_file is unspecified, the output should be written to standard output.

Preview additional formats

Adding a preview function is very simple. All you need to do is define a function that returns preview information, and add an entry point to link the function to certain file format.

More specifically, a previewer should be specified as

pattern,priority = preview_module:func

or

module:func,priority = preview_module:func

where