Automations - Rebase Platform Documentation

Rebase provides two way to automate the execution of data pipelines: schedules and event-based triggers:

Schedules: Starts a job at a specified time.
Event-based triggers: Starts a job when a specific event occurs.

Schedules

A schedule allows to execute a specific pipeline at a fix interval. To run a Python function with a fix interval of one day, we add the @pipeline decorator to the function with a schedule parameter according to:

# schedule.py
import rebase as rb
from rebase.pipelines import pipeline

@pipeline(schedule=rb.Period(days=1))
def scheduled_run():
    print("It's one day later than previous run!")

To instead schedule the run at a specific clock time, we can use the cron syntax. A nice tool for finding the cron expression that matches your needs is crontab.guru. Here is an example of a pipeline scheduled at a specific clock time:

# schedule.py
import rebase as rb
from rebase.pipelines import pipeline

# Runs at 11 pm (UTC) every Monday
@pipeline(schedule=rb.Cron("0 23 * * 1"))
def scheduled_run():
    print("It's Monday at 11 pm!")

To make the scheduled function into a deployment, run the following command using the CLI:

rebase deploy --name my_schedule schedule.py

Or equivalently in a programmatic way:

if __name__ == "__main__":
    rebase.runner.deploy_app(scheduled_run)

Event-based triggers