Skip to content

Template Developer Guide

A step-by-step guide for getting your processing app running on Dory — from an empty directory to a published template processor that subscribers can reach.

The Big Picture

Dory streams events (from cameras and sensors) to processing apps. Your app is one of those processors. Dory does not run your code directly — it runs it as a Docker container.

Publishing a template is five things:

  1. Create the template, which also issues its publishing credentials.
  2. Generate a project skeleton.
  3. Build it as a container image.
  4. Upload that image to Dory's registry.
  5. Declare the event types it emits, attach it to a lens, and submit it for review.

Steps 4 and 5 are the ones people get wrong. An image with no declared event types publishes cleanly and then matches no subscription — the subscriber's coverage report says why, but nothing tells you.

Subscribers reach your template through a lens and a set of regions. The regions select sensors — by the area each sensor watches, not by anything your code does — and your template is chosen per sensor from the lens, by the sensor types and event types its live version declares. The location you stamp on each event travels to the subscriber but does not decide who receives it. See Stamping a location and How a subscription reaches your template.

A version does not go live when you submit it. A Dory administrator reviews the submission and approves it, and only then does it run against any sensors. That is why the event types, the sensor types and the lens are all settled before you submit: the reviewer is approving the whole configuration, not just the image.

The portal walks you through all of this. At portal.streetscapes.ai → Processor Templates → Create template, an eight-step flow does every browser-side step for you and hands you the exact terminal commands, pre-filled with your own template's slug. Shipping a later version of a template that already works uses the same dialog in a shorter five-step form. This guide is the same path written out, and the reference for scripted setups.

The portal's steps map onto this guide as: Create template → step 1, Get publisher credential → step 1's credentials, Install libraries → the prerequisites, Initial setup → step 2, Build version container → step 3, Upload version container → step 4, Submit for review → step 5, Summary → step 6.

Prerequisites

  • Docker installed and running
  • The AWS CLI installed (used for the registry login; you do not need an AWS account)
  • Python 3.11 or newer — the processor SDK requires it
  • The publishing CLI and its one dependency:
python3.11 -m venv .venv && source .venv/bin/activate
pip install requests
curl -O https://portal.streetscapes.ai/downloads/dory_util.py

Every command below runs from the directory you just downloaded dory_util.py into. Stay there for the whole guide — the project gets created in a subdirectory, and nothing requires you to change into it.

Global flags go before the subcommand. --debug, --base-url, --token, --client-id, --client-secret, --scope and --header are declared on the top-level parser, so dory_util.py --debug push-image … works and dory_util.py push-image --debug exits with "unrecognized arguments".


Step 1: Create the Template

This tells Dory "a processor called my-template exists", and issues the credentials that will publish it.

In the portal (recommended): Processor Templates → Create template. Enter a name; the slug is derived and shown before you commit to it.

From the CLI, with an admin user token:

export DORY_ACCESS_TOKEN="<your admin token>"

python3 dory_util.py create \
  --name "My Template" \
  --slug my-template

The slug is your processor's permanent ID — lowercase, hyphenated, and it cannot be changed or released afterwards. It becomes the registry path and the name of the publishing credential.

The response's credentials object holds a clientId and clientSecret. These are the publishing credentials — they are created with the template automatically. You do not create them separately.

The secret is permanent and cannot be rotated. Store it in a secrets manager, not in a repository. If you lose your copy, open the template in the portal and copy it again from its Publisher credential section, provided your role grants both app_clients:write and processor_template_versions:write.

You cannot mint publisher credentials by hand — dory_util.py m2m-create deliberately refuses any name matching template-<slug>-publisher, because the mapping is established at creation.


Step 2: Generate the Project

python3 dory_util.py scaffold \
  --name "My Template" \
  --slug my-template \
  --path ./my-template \
  --ci github

That writes a complete, buildable project into ./my-template:

File What it is
main.py A BaseProcessor with startup / run / shutdown, and a guarded publish() example
dory.yaml SDK settings, with health_port pinned to 8080
requirements.txt The SDK, pinned — dory-processor-sdk[production]==0.0.6
Dockerfile Pinned to linux/amd64, installs from requirements.txt
.env.example Your app's own settings — the event type and a placeholder location — plus, commented out, the SDK and broker variables for a local test. What the platform injects for real is listed under What your container receives
README.md Your publish commands, with your slug already filled in
tests/ A smoke test
.github/workflows/ A manual-dispatch publish workflow (with --ci github)

Now edit main.py — replace the body of run() with your own work, and the placeholder location (next). Everything else is already correct.

Drop --ci github if you do not want a workflow. Pass --force to overwrite an existing project.

Stamping a location

The one publish call is await self.publish(event_type, location, payload) — see Output & Events. The GeoPoint you pass is the only source of the event's location: the SDK encodes it into a nine-character geohash, one character per segment, and appends it to the routing key after the processor id and the event type. Nothing else in the platform sets, corrects or replaces it.

The scaffold stamps a constant placeholder — PROCESSOR_LAT / PROCESSOR_LNG, defaulting to 26.3683 / -80.1289 — on every event. Replace it before you ship. The sensor your container is bound to is described in the environment it receives: its location as a geohash in DORY_GEOHASH, its metadata in DORY_SENSOR_METADATA. For a position per detection, use the SDK's CameraGeolocalizer with the calibration points in DORY_GEO_REFERENCE_POINTS (at least four for a camera view), and fall back to a fixed camera location when there are none — which is what the SDK's reference camera example does.

The stamp is carried, not matched

A subscription's regions select sensors by their area of view, and every event your processor publishes for the requested event types is delivered whole. The geohash you stamp reaches the subscriber in the routing key, but nothing filters on it. A wrong stamp misplaces your events for every subscriber, and nothing corrects it; a right stamp does not narrow who receives them.

flowchart LR
    T["Your template<br/>publish(event_type, location, payload)"]
    S["Processor SDK<br/>encodes the location → 9-character geohash<br/>prefixes the processor id"]
    K["Routing key<br/>‹processor id›.person.d.h.x.p.f.r.5.b.7"]
    X["Event exchange"]
    Q["Subscriber's queue<br/>bound on ‹processor id›.person.#"]
    N["The nine geohash segments ride along —<br/>nothing matches on them"]
    T -->|"location: yours, unchanged"| S
    S -->|"builds"| K
    K -->|"published"| X
    X -->|"routed on processor id + event type only"| Q
    K -.- N

You choose the location; the SDK only encodes it. The subscriber's queue is bound on the processor id and the event type, so every event your processor publishes for that type is delivered, whatever you stamped.

Bringing your own project

If you already have an app, you need a Dockerfile that targets amd64 and installs the SDK:

FROM --platform=linux/amd64 python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY main.py dory.yaml ./
EXPOSE 8080
CMD ["python", "main.py", "--config", "dory.yaml"]

with dory-processor-sdk[production]==0.0.6 in requirements.txt. Computer-vision apps need system libraries the slim image lacks — add before the pip install:

RUN apt-get update && apt-get install -y --no-install-recommends \
    libgl1 libglib2.0-0 ffmpeg && rm -rf /var/lib/apt/lists/*

Prefer opencv-python-headless over opencv-python, and if you use PyTorch install the CPU-only wheels (--index-url https://download.pytorch.org/whl/cpu) — the default wheel bundles ~5GB of CUDA libraries the container cannot use.


Step 3: Build for linux/amd64

docker buildx build --platform linux/amd64 \
  -t my-template:latest --load ./my-template

--platform linux/amd64 is not optional. Dory runs processors on amd64 hosts, and nothing validates architecture when you push. An arm64 image built on an Apple Silicon Mac uploads fine, registers fine, and then fails at startup with exec format error.

The build context is ./my-template, so there is no need to change directory — and staying put keeps dory_util.py within reach for the next step.

To try it locally first:

(cd my-template && DORY_HEALTH_PORT=8080 python main.py)
curl -s localhost:8080/health

The parentheses keep the directory change inside a subshell. A no publisher configured; would emit … log line is expected locally — the platform supplies the broker settings and a publish-only credential when it runs your container for real (see What your container receives).


Step 4: Push the Image

Switch from your admin identity to the publisher identity:

unset DORY_ACCESS_TOKEN

export DORY_CLIENT_ID="<clientId from Step 1>"
export DORY_CLIENT_SECRET="<clientSecret from Step 1>"

python3 dory_util.py push-image \
  --tag v1.0.0 \
  --local-image my-template:latest

unset DORY_ACCESS_TOKEN comes first because a user token outranks client credentials, and the push refuses a user token.

Do not set DORY_SCOPE. Left unset, the token carries every scope the credential holds; requesting a scope it does not hold makes Cognito reject the whole request with invalid_grant.

No # comments in that block, deliberately. zsh leaves INTERACTIVE_COMMENTS off, so a pasted line beginning with # fails with zsh: bad pattern: # before the command that mattered ever runs. The portal's Upload version container step emits the same comment-free block, and its copy button fills in your publisher secret for you — the secret is masked on screen and real on the clipboard, so the block pastes and runs as-is.

You do not need to create the container registry first — Dory creates it on your first push.

This uploads the image but deliberately does not register a version yet. That happens in Step 5, together with the event types and the lens, so the version is complete when a reviewer sees it rather than being submitted empty and amended afterwards.


Step 5: Declare Event Types, Attach a Lens, and Submit for Review

This is the step people miss. Three declarations happen here, and the version is submitted with all three:

  • Event types (supportedTags) — what your processor emits. This determines which subscriptions a lens containing your template can serve. A version with none can never match anything; the subscriber's coverage report will say so (NO_TEMPLATE_EMITS_IT), and nothing tells you. You do not need to emit every type a subscriber asks for: per sensor, the platform picks the smallest set of templates in the lens whose declared types cover the request, so a template declaring one type is chosen alongside another declaring the rest.
  • Sensor types (supportedSensorTypes) — what it accepts. A template is paired only with sensors of a type it declares, so a lidar processor left as camera is matched against the wrong sensors — and a subscriber whose matched sensors are of another type sees NO_COMPATIBLE_SENSOR.
  • A lens — how subscribers reach it. A lens is a named group of processors that subscribers reference by alias, and an unattached template is invisible to every subscription. A lens may be public: any organization may then subscribe through it, and only public templates can be attached to it (see Lenses).

In the portal: the guided flow's Submit for review step has all three on one screen — your organization's event types as checkboxes, the sensor types, and the lenses you can attach to. Event types are picked there by name, so you never handle their ids; a publishing credential cannot list them. Submitting is refused until at least one event type and at least one lens are selected, because a version approved without either goes live and still reaches nobody.

From the CLI, if you already know the UUIDs:

python3 dory_util.py raw --method POST --path /template-processors/self/versions \
  --json-body '{"tag":"v1.0.0","supportedTags":["<tag-uuid>"],"isActive":true}'

All three declarations can travel in that one request — sensor types as supportedSensorTypes, which, left out, is copied from the template's latest version, or is camera for its first — and it is applied as one transaction: the version and its routing either both land or neither does. An optional lensIds array sets the exact set of lenses the template should belong to:

  --json-body '{"tag":"v1.0.0","supportedTags":["<tag-uuid>"],"lensIds":["<lens-uuid>"]}'

Omit lensIds and lens membership is left exactly as it is. That is why the publisher credential, publish_template_version.sh and the generated GitHub workflow — none of which knows about lenses — are unaffected.

lensIds needs the lenses:write permission, but only when it actually changes something: sending back the lenses a template already belongs to writes nothing and is allowed for any publisher. A publishing credential holds no lens permission, so a change from one is refused — attach the lens on the portal's Lenses page, or in the publish wizard, before you submit.

"isActive": true in that body is inert. Registering a version is a submission: the database refuses to mark any version live whose review status is not approved, so nothing the client sends can shortcut the review.

Without the event types, a subscriber's subscription is still accepted and goes active — but its coverage report marks your event type not covered with the reason NO_TEMPLATE_EMITS_IT. Once a version declaring the type is approved, the hourly re-check picks it up and the subscriber is told the coverage changed.


Step 6: Wait for Review, and Verify

A Dory administrator reviews the submission. They see the image and its digest, the event types and sensor types the version declares, and every lens the template is routed through — which is why all of that is settled in Step 5.

Three outcomes:

Outcome What it means
Approved The version becomes the live one at once, and the version it replaced is stood down in the same step. Processors started from then on run it; approval does not restart a processor that is already running. A new subscription can use your template straight away, and an existing one it can now serve picks it up at its next hourly re-check
Rejected The version stays registered and does not run. Address the feedback and push a new tag — an administrator can approve a rejected version later, but do not plan on it
Revoked The approval is withdrawn. If it was the live version, the template has no live version until another approved one is made current: no new processor starts from it, and processors already running it are stopped at the orchestrator's next check. At the next hourly re-check the subscriptions it served drop your template — those event types stay covered only if another template in the lens emits them — and pick it up again once another approved version is live

A rejection always carries written feedback, and a revocation may; it is shown when you open the template in the portal.

Once approved, confirm:

python3 dory_util.py versions --slug my-template

You should see v1.0.0 listed and active.

This verifies configuration, not execution. Per-processor runtime status is not currently exposed to organizations, so a container that starts and immediately crashes looks the same here as one running happily.


How a subscription reaches your template

flowchart TB
    SUB["Subscription<br/>event types · regions · lens"]
    L{"Lens still usable?<br/>the subscriber's own, or public"}
    NONE["No processors<br/>coverage: LENS_NOT_ACCESSIBLE"]
    SEN["Sensors<br/>the subscriber's own + any public ones · active ·<br/>area of view intersects a requested region"]
    TPL["Templates in the lens<br/>with an approved live version"]
    FIT["Keep those declaring the sensor's type<br/>and at least one requested event type"]
    COVER["Per sensor: the smallest set of templates<br/>whose declared event types cover the request"]
    PAIR["One processor per (sensor, template)<br/>reused if already running, started if not"]
    REP["Coverage report to the subscriber<br/>per event type: covered, or why not"]
    SUB --> L
    L -->|no| NONE
    L -->|yes| SEN
    SEN --> TPL --> FIT --> COVER --> PAIR --> REP

Regions never touch your template directly — they pick sensors by what those sensors watch. Your template is chosen per sensor from the lens, by the sensor types and event types its live version declares, and the same check runs again every hour for every active subscription.

A subscription names three things: the event types it wants, the regions it wants them from, and a lens. Matching runs in this order:

  1. The lens gate. The lens must still exist and be the subscriber's own or public. This is re-evaluated on every run — if another organization's lens is made private, or a lens is deleted, the subscription stays active with nothing matched; a lens made public again is matched again at the next re-check.
  2. Sensors. The subscriber's own sensors plus any organization's public ones, active and not decommissioned, whose area of view — the polygon a sensor watches, not the point it stands on — intersects a requested region. A region of * means everywhere the subscriber can see.
  3. Templates. Those in the lens with an approved live version, kept if they declare the sensor's type and at least one of the requested event types.
  4. The smallest covering set. Per sensor, the fewest templates whose declared event types cover the request, one processor each. Every requested type is kept, so a template declaring one type is chosen alongside another declaring the rest.
  5. Processors. One per (sensor, template) pair — reused if one is already running, started if not. Processors are shared between subscriptions, so one container of yours may serve several subscribers. And because sensors are matched for the subscriber, your container may read a sensor that is not your organization's: another organization's public sensor, or, through a public lens, the subscribing organization's own.

The subscriber is told the outcome as a coverage report: for each requested event type, whether it is covered and, if not, why — LENS_NOT_ACCESSIBLE, NO_SENSORS_IN_REGION, NO_TEMPLATE_EMITS_IT or NO_COMPATIBLE_SENSOR — plus four counts: sensors matched, sensors served, processors started and processors reused. It never names sensors or processors, so a subscriber cannot tell which template is serving them. Nothing matching is not an error: the subscription is live with an empty report until the world changes.

Timing. A new subscription is matched when it is provisioned. Every active subscription is re-checked hourly, so attaching or detaching your template, approving a version, or revoking one reaches existing subscriptions at their next re-check; the subscriber is told only when their coverage actually changed. Revocation does not wait for the re-check to stop your code: if the revoked version was the live one, processors running your template are stopped at the orchestrator's next check, and the re-check then puts your template back on those subscriptions only if another approved version has been made live.

What your container receives

When the platform starts your container for a sensor, it sets these variables:

Variable What it holds
PROCESSOR_ID This processor's identity — the first segment of every routing key it publishes
DORY_SENSOR_ID The sensor this processor is bound to
DORY_SENSOR_TYPE The sensor's type — camera, lidar, …
DORY_SENSOR_CONNECTION_CONFIG How to reach the sensor, as JSON — never a login. For a camera, its stream_url points at the platform, which signs in to the camera for you. Keep it out of logs and events: the sensor may be another organization's
DORY_SENSOR_METADATA The sensor's metadata, as JSON
DORY_GEO_REFERENCE_POINTS The sensor's calibration points — set only when the sensor has any
DORY_GEOHASH The sensor's own location as a nine-character geohash — set only when the sensor has a location. Informational: publish() never reads it

plus the broker settings and the credential to publish with.

That credential is shared and platform-managed, and it can only publish. A processor cannot declare queues, bind, or read, so a template can never consume other processors' events and never configures the broker. The DORY_RABBITMQ_* names in .env.example exist so a local run can point at a broker of your own; in production the platform supplies them.


Publishing an Update

After the one-time setup, a release is:

docker buildx build --platform linux/amd64 \
  -t my-template:latest --load ./my-template

python3 dory_util.py push-image \
  --tag v1.1.0 \
  --local-image my-template:latest

Then declare the event types for the new version in the portal and submit it, as in Step 5. Every version is reviewed, not just the first — the one that is live keeps serving until the new one is approved.

In the portal, open the template and choose New version on its Versions tab, which opens the same flow in a shorter five-step form: Get publisher credential, Build version container, Upload version container, Submit for review, Summary. It carries the previous version's event types and sensor types forward, and the lens the template already belongs to is shown on the Submit for review step so you can confirm or change it before the reviewer sees it. It skips creation, the library install and the scaffold — the scaffold in particular would regenerate the project over the edited main.py that is the new version.

Use a new tag each release — re-registering an existing tag creates a second version row carrying the same label, and a second review for the same image.

To roll back, open the template's Versions tab in the portal and choose Make live on an earlier version. Only an approved version can be made current.

Deactivating a template, or clearing its current version, stops every running processor built from it at the orchestrator's next check, and those processors are marked terminated rather than paused. Making a version current again does not revive them: the next hourly re-check starts fresh processors for every subscription that still wants them, and until then those subscriptions receive nothing from your template.


Publishing from CI (GitHub Actions)

scaffold --ci github already wrote .github/workflows/publish-template-version.yml. It is manual-dispatch by design — publishing on every merge mints a version, and a registry image, for changes that were never meant to ship.

Add two repository secrets, DORY_CLIENT_ID and DORY_CLIENT_SECRET. Then run the workflow from the Actions tab with the tag you want to publish, and declare its event types in the portal afterwards. The workflow pushes an image; it cannot submit a version for review or attach a lens, so the portal step is still required.


Troubleshooting

Error Likely cause
can't open file '…/dory_util.py' You changed into the project directory. The CLI lives one level up, where you downloaded it — cd ..
unrecognized arguments: --debug Global flags go before the subcommand: dory_util.py --debug push-image …
This endpoint requires an M2M app client token You're pushing with a user token — unset DORY_ACCESS_TOKEN and use the publisher credentials
Path slug 'self' is only valid for M2M app client tokens Same as above — push-image only works with publisher credentials
Image not found in ECR for provided tag The version is being registered before the image was pushed. Run Step 4 first
Token request failed Wrong DORY_CLIENT_ID / DORY_CLIENT_SECRET. Do not set DORY_SCOPE to fix this
Template publisher credentials are created with the template and cannot be created here Publisher credentials come from Step 1; open the template in the portal and copy them from its Publisher credential section
Required command not found: aws / docker Install the AWS CLI / Docker
Push succeeds but the processor never starts (exec format error) The image is arm64 — rebuild with --platform linux/amd64 and push again
A subscriber's coverage report says an event type is not covered — NO_TEMPLATE_EMITS_IT No template in the lens with a live version declares that event type. Check supportedTags on your live version — see Step 5
… NO_COMPATIBLE_SENSOR A template emits the type, but none of the matched sensors is of a type your live version declares. Check supportedSensorTypes
Subscribers receive your events but they appear in the wrong place The location you pass to publish() is what they see — the scaffold's placeholder is still in main.py. See Stamping a location
Your container starts but cannot read from a queue By design: the credential a processor gets can only publish — see What your container receives
zsh: bad pattern: # when pasting a block You pasted a # comment into zsh, which leaves INTERACTIVE_COMMENTS off. The blocks in this guide and in the portal carry no comments for exactly that reason
The version is registered but nothing runs It is awaiting review, or was rejected. Open the template in the portal to see its review status and any reviewer feedback
A version cannot be set as current Only an approved version can be made live — the database refuses any other pairing

Add --debug before the subcommand to see the full HTTP exchange. It redacts the request's Authorization header but prints response bodies in full, so avoid pairing it with commands that return credentials.