Non-packaged vs Packaged¶
Soopervisor supports two types of projects: non-packaged and packaged.
Non-packaged are simpler projects that require fewer configuration files. They
only need a
pipeline.yaml file to be valid. Non-packaged projects are a
good option for small projects. To create one:
# create a base non-packaged project ploomber scaffold
Packaged projects have more structure and require more configuration files. The main advantage is they allow you to organize your work better.
For example, if you have some Python modules that you reuse in several files,
you must to modify your
sys.path to ensure that such
modules are importable. If your project is packaged, this isn’t necessary,
since you can install your project with pip:
pip install --editable path/to/myproject
After installation, you can import modules from your project anywhere in a Python session, notebook, or other modules inside your project, making it simpler to create modular code.
To create a base package project:
# create a base packaged project ploomber scaffold --package