Consuming Events¶
How events reach your code from the moment subscribe() returns — and what else travels in the same queue.
The callback model¶
Every subscription has a callback — a function you pass to subscribe() that runs once per delivered event. Only events reach it: the platform's own status messages travel in the same queue, and the SDK handles those itself (see Events and control messages below).
def handle_event(event: dict):
# event is the parsed JSON envelope
message_id = event["message_id"] # unique per event — de-duplicate on it
payload = event["payload"] # whatever the template behind your lens emitted
print(f"{event['timestamp']} {message_id}: {payload}")
The SDK decodes each message body from UTF-8, parses it as JSON, and calls your function with the resulting dict. Every event has the same outer shape — schema_version, message_id, timestamp and payload. Only payload varies: it is whatever the template behind your lens emits, so treat its fields defensively (event["payload"].get(...)).
The event type and the position stamped on an event are not in the envelope. They travel on the message's routing key, which the SDK does not pass to your callback. When you need to tell event types apart, run one subscription per event type, each with its own callback (see Multiple subscriptions), or use a template that includes the type in its payload.
Events and control messages¶
Your delivery queue carries two kinds of message. Events come from the processors matched to your subscription. Control messages come from the platform itself, and tell you how the subscription is doing:
| Message | When it arrives | What it carries | What the SDK does with it |
|---|---|---|---|
ready |
Provisioning finished | The coverage report | Marks the subscription active and stores the report; wait_until_ready() returns it |
updated |
Each time an hourly re-check finds your coverage different — a sensor installed, published, unpublished, moved or decommissioned; a template attached, detached or revoked; the lens made private, deleted or reopened | The new coverage report | Replaces the stored report |
error |
The platform gave up setting the subscription up | A code (BROKER_REFUSED or PROVISIONING_FAILED) and message, a plain sentence saying what to do |
Marks the subscription error and stores the code and sentence; wait_until_ready() raises SubscriptionError |
A control message is marked by a message property (AMQP type, starting with dory.subscription.) that no processor ever sets, which is how the SDK tells the two apart. None of them reach your callback. The SDK prints one line for each, ignores any kind it does not know, and keeps a local view current that you can read at any time without a network call:
client.rabbitmq.subscription_status(sub_id) # {"status": ..., "coverage": ..., "error": {"code", "message"} or None}
client.get_subscription(sub_id) # the platform's record: status, coverage, lastErrorCode — one request
updated is not a one-off: expect one every time a re-check changes your coverage, for as long as the subscription lives. A re-check that changes nothing sends nothing. A ready can occasionally arrive a second time; the SDK simply records it again. What the coverage report contains is described in Subscriptions & Events; how to wait for it, and what each error code means, is in Managing Subscriptions.
sequenceDiagram
participant P as Processors (matched sensors)
participant Q as Your delivery queue
participant W as Platform
participant S as SDK consumer thread
participant H as Your callback
S->>Q: start consuming (as subscribe() returns)
W->>Q: bind to your processors
P-->>Q: event (no type property)
Q->>S: event
S->>H: dict — schema_version, message_id, timestamp, payload
Note over Q,H: can happen before ready
W-->>Q: dory.subscription.ready {coverage}
Q->>S: control message
S->>S: status = active, store coverage — wait_until_ready() returns
W-->>Q: dory.subscription.updated {coverage} — every time coverage changes
Q->>S: control message
S->>S: replace the coverage report
Note over S,H: control messages never reach the callback
Events can arrive before ready. The platform binds your queue to its processors first and sends ready after, and a processor may already be running for another subscription — so your callback may fire while wait_until_ready() is still blocking. Waiting is optional; events that arrive while you wait are delivered normally.
Background threads¶
subscribe() is non-blocking. Consuming begins as it returns — the queue exists from the moment the platform accepted the subscription. Each subscription consumes its queue on its own background daemon thread, so your main thread must stay alive to keep receiving events:
Because consumers run on daemon threads, they stop when your program exits. If events "stop arriving," check these in order:
- The main thread ended. Daemon threads die with it.
- The connection dropped. The SDK prints an error and the consumer thread ends — it does not reconnect. The subscription itself is still alive on the platform, so
unsubscribe()it by its id before you subscribe again (or restart the process): subscribing again without that creates a second subscription and leaves the first one running. - The subscription was deleted elsewhere — from the portal, or by another process. Deleting a subscription removes its queue at once; the broker then cancels the consumer and the thread ends quietly: nothing is printed, and
subscription_status()still shows the last status it heard.client.get_subscription(sub_id)raisesSubscriptionNotFoundfor it. - The subscription is not delivering. It may still be
pending, it may beerror(the platform has stopped setting it up — delete it and subscribe again), or it may beactivewith a coverage report that says nothing matched. A processor the platform had to start for you sends nothing until it is running. Check withwait_until_ready()orclient.get_subscription(sub_id).
flowchart LR
S["subscribe() returns the id"] --> R["Consumer thread reading your queue"]
R -->|"unsubscribe() or close()"| E1["Thread stopped,<br/>then the subscription is deleted"]
R -->|"queue deleted elsewhere<br/>(portal or another process)"| E2["Broker cancels the consumer,<br/>thread ends silently"]
R -->|"connection dropped"| E3["Error printed, thread ends —<br/>no automatic reconnect"]
Keep callbacks quick and thread-safe
Your callback runs on the consumer thread. Long-running work blocks further delivery for that subscription — hand heavy work off to a queue or worker. If you touch shared state from multiple subscriptions' callbacks, guard it.
Delivery semantics¶
- At-least-once. A message is acknowledged after your callback returns, so anything the consumer had taken but not yet acknowledged when your process died can be delivered again. Key on
message_idand ignore an id you have already processed. - Errors are swallowed, then acked. If your callback raises, the SDK prints the error and still acknowledges the message — it is not redelivered. Do your own error handling inside the callback if you need retries.
- Prefetch. The consumer takes ten messages at a time, so a slow callback applies backpressure rather than pulling the whole queue into memory.
flowchart TD
M["Message delivered to the consumer thread"] --> T{"AMQP type starts with<br/>dory.subscription. ?"}
T -->|"yes: control"| C["Update the subscription's local status<br/>ready / updated / error"] --> A1["acknowledge"]
T -->|"no: event"| D["Decode UTF-8, parse JSON"]
D --> CB["Call your callback with the dict"]
CB -->|"returns"| A2["acknowledge"]
CB -->|"raises"| L["Print the error"] --> A3["acknowledge anyway —<br/>not redelivered"]
- Retention. A message waits in your queue for up to an hour, and at most 10,000 wait at once; beyond that the oldest are dropped. Control messages share the same limits, so a
readynobody read in time is gone.wait_until_ready()covers that, and a consumer thread that died before the message arrived: when nothing comes within its timeout, it asks the platform once before giving up. - Where an event happened. The position stamped on a message is the point the template chose to record — a fixed location, or a per-detection estimate — and the platform does not filter events by it. Your regions select sensors, by area of view; every event of your requested types from the processors matched to your subscription is delivered, including ones stamped outside your region. If you need per-event location filtering, do it on a location the template's payload carries, if it carries one.
flowchart LR
G["Your regions"] -->|"sensors whose area of view<br/>overlaps a region"| S["Matched sensors"]
S -->|"one processor per<br/>sensor and template"| P["Processors"]
P -->|"every event of your event types,<br/>position stamped by the template"| Q["Your delivery queue"]
Q -->|"no per-event region filter"| H["Your callback"]
Older SDKs
On SDK versions before 1.2.0 a control message is handed to your callback as if it were an event. It has the usual outer shape, but its payload holds subscription_id and status (with a coverage report, or an error code and sentence) instead of anything a template emitted. Upgrade, or guard the handler against it.
A complete consumer¶
import json, signal, time
from dory import DoryClient, SubscriptionError, SubscriptionTimeout
running = True
seen: set[str] = set() # bound this in real code — it grows with every event
def handle_event(event: dict):
message_id = event["message_id"]
if message_id in seen:
return # delivered again — already handled
seen.add(message_id)
print(json.dumps(event["payload"], indent=2))
def stop(*_):
global running
running = False
signal.signal(signal.SIGINT, stop)
signal.signal(signal.SIGTERM, stop)
client = DoryClient()
sub_id = client.rabbitmq.subscribe(
event_types=["person", "moving"],
geohash_regions=["dhxpfr5"],
lens_alias="demo-lens",
callback=handle_event,
name="demo-consumer",
)
# Optional: events may already be arriving while this waits.
try:
coverage = client.rabbitmq.wait_until_ready(sub_id, timeout=120)
print("Coverage:", json.dumps(coverage, indent=2))
except SubscriptionTimeout:
print("Still provisioning — events start when it finishes")
except SubscriptionError as err:
print(f"Subscription failed ({err.code}): {err.message}")
running = False # it will not be set up; close() deletes it
try:
while running:
time.sleep(1)
finally:
client.close() # unsubscribes everything this client created
wait_until_ready() raises SubscriptionError when the platform gave up setting the subscription up. err.code names why and err.message is the plain sentence for that code. The platform will not finish setting that subscription up, so delete it and subscribe again — here client.close() deletes it. The codes are explained in When a subscription is in error.