Skip to content

Getting Started

The Dory Processor SDK lets you build stateful Python processors that survive pod migrations, graceful shutdowns, and node drains with their in-memory state intact. You write the business logic; the SDK and platform handle lifecycle, state hand-off, health, and observability.

This page walks the whole journey: write a processor, run it locally, package it as a container, register it as a template, and — once it is approved — let the platform run it wherever a subscription needs it.

flowchart LR
    A["Write processor<br/>(SDK)"] --> B["Run locally<br/>(python main.py)"]
    B --> C["Package<br/>(container image)"]
    C --> D["Register & submit<br/>(template version, lens)"]
    D --> R["Approved<br/>(administrator review)"]
    R --> E["Runs per sensor<br/>(when a subscription matches)"]

Package naming

The package installs as dory-processor-sdk but imports as dory. All examples use from dory import ... / from dory.simple import ....

1. Install

pip install dory-processor-sdk

For container/Kubernetes deployments, install the production extra (it pulls in the runtime dependencies the platform needs):

pip install "dory-processor-sdk[production]"
Extra When to use
[production] Building a container image to deploy on the platform
[dev] Linting and test tooling for local development

2. Scaffold a project

If you intend to publish this processor to Dory, use the publishing CLI instead. It generates a project that is ready to build and publish — SDK pinned, Dockerfile targeted at linux/amd64, and a README carrying your own template's publish commands:

curl -O https://portal.streetscapes.ai/downloads/dory_util.py
python3 dory_util.py scaffold --name "My Processor" --slug my-processor --path ./my-processor

The Template Developer Guide walks the whole path.

For a bare SDK starter with no publishing scaffolding, the SDK ships its own command:

dory init my-processor
Flag Meaning
-o DIR Output directory. Without it files land in the current directory — dory init my-processor does not create a my-processor/ folder
-f Overwrite existing files

dory init generates a starter main.py (a BaseProcessor with stateful attributes and startup/run/shutdown hooks) and a Dockerfile.

Two limits worth knowing: it writes only those two files, and its Dockerfile neither pins the SDK version nor sets a platform — so build with docker buildx build --platform linux/amd64, or an image built on an Apple Silicon Mac will fail on the platform with exec format error.

3. Write a processor

The minimal "hello processor" is a class with a run() loop. stateful(...) declares attributes that are automatically saved and restored across restarts and migrations.

from dory import DoryApp, BaseProcessor, stateful

class CounterProcessor(BaseProcessor):
    counter = stateful(0)
    seen = stateful(dict)        # use a factory (dict/list) for mutable defaults

    async def startup(self) -> None:
        self.context.logger().info("starting up")

    async def run(self) -> None:
        async for i in self.run_loop(interval=1.0):
            self.counter += 1

    async def shutdown(self) -> None:
        self.context.logger().info(f"final counter={self.counter}")

if __name__ == "__main__":
    DoryApp().run(CounterProcessor)

DoryApp().run(...) takes your processor class (not an instance) and drives the whole lifecycle. See Core Concepts for the lifecycle and what stateful does.

config() and logger() are methods

context.config() and context.logger() are methods — call them with ().

Function-based alternative

For simple processors, the dory.simple API skips the class. The @processor decorator collects module-level state(...) variables and starts the app automatically when the module runs as __main__:

from dory.simple import processor, state

counter = state(0)

@processor
async def main(ctx):
    async for i in ctx.run_loop(interval=1.0):
        counter.value += 1

Import dory.simple directly

from dory.simple import processor, state — these are not re-exported as from dory import .... The @processor decorator must be at module top level.

4. Run locally

Run either form directly with Python:

python main.py

The SDK starts a health server, runs startup(), restores any saved state, then enters your run() loop. Liveness (/health) and readiness (/ready) endpoints and a Prometheus /metrics endpoint are served on the health port (default 8080). Press Ctrl+C to trigger a graceful shutdown and a final state save.

You can also validate your configuration before running:

dory validate           # validate default config
dory validate -c FILE   # validate a specific config file

5. Package as a container

Build the image from the generated Dockerfile:

docker build -t my-processor:0.1.0 .
docker run -p 8080:8080 my-processor:0.1.0

Then push it to your container registry.

6. Register a template and let the platform run it

Register your image as a processor template version — declaring the event types it publishes and the sensor types it accepts — attach the template to a lens, and submit the version for review. A registered version does not run anywhere on its own:

  • An administrator approves it first. Until a Dory administrator approves the version, it does not run against any sensor.
  • Subscriptions decide where it runs. When a subscription names a lens that holds your template and asks for an event type your version publishes, the platform picks sensors in the requested regions by each sensor's area of view — of a sensor type your version accepts — and runs one processor — one container of your image — per sensor and template, reusing a live one for that pair if there is one. Processors are shared between subscriptions, and a processor is stopped once the last subscription using it is released.
  • The platform routes your events. Each subscriber's queue is bound to your processor by the event types that subscriber requested; your code only calls publish. See Output & Events.

Within that, the platform provisions the pods and migrates state when nodes change, without changes to your code. The Template Developer Guide walks through registration, review and matching.

Production config is injected for you

In production the platform injects configuration (pod identity, state backend, timeouts, orchestrator URL) through the environment. You do not hand-write production config. See Configuration for the env var reference.

Next steps