Priostack · Engineering Blog · 5 April 2026 · 18 min read

BPM + Integration: The Two-Layer Architecture Behind Priostack

Architecture Engineering Deep dive

A common question when evaluating Priostack: "What happens when I need more than a job worker fetching tasks over REST? What if I need deduplication, content-based routing, or message correlation across multiple process instances?"

The answer is a two-layer architecture. Layer 1 is the BPM runtime - BPMN, DMN, CMMN execution with a clean REST job API. It is the complete runtime for the majority of use cases. Layer 2 is an optional Enterprise Integration Patterns (EIP) layer built into the same deployment: no broker, no extra service, just configuration. The two layers are entirely independent by design: the integration layer knows nothing about processes, and the process runtime knows nothing about the integration layer.

This article walks through both layers in depth: what each one gives you, where the boundary sits, and - critically - how the integration layer makes idempotency, deduplication, and message orchestration a configuration problem rather than an infrastructure one.

1. The stack at a glance

2. Layer 1 - The BPM runtime

Layer 1 is the engine. It covers three process notations:

The REST job API

The fundamental interaction model is pull-based. Workers are external services - a Python microservice, a Node script, a Go binary, anything that speaks HTTP. They operate a three-step loop:

# 1. Activate  -  fetch a locked job from the engine
POST /api/v1/jobs/activate
     { "type": "credit-check", "maxJobsToActivate": 1, "worker": "risk-svc" }
→    { "jobs": [ { "key", "type", "processInstanceKey", "variables", ... } ] }

# 2. Execute  -  your logic runs locally, fully isolated
# The engine waits. The job is LOCKED with a timeout.

# 3. Complete or fail
POST /api/v1/jobs/{key}/complete   { "variables": { "approved": true } }  → 204
POST /api/v1/jobs/{key}/fail       { "errorMessage": "timeout", "retries": 2 }

No broker. No persistent connection. No SDK required. If the worker dies between steps 2 and 3, the job lock timeout expires and the job re-enters the queue automatically.

Instance state and observability

Every process instance has a state maintained by the engine: ACTIVE, INCIDENT, COMPLETED, or TERMINATED. Incidents are raised automatically when a job exhausts its retries or when a worker calls the fail endpoint with retries remaining at zero. From the dashboard you can see exactly which element in the BPMN graph triggered the incident, inspect the variables at that point, retry from there, or cancel the instance.

3. Layer 2 - The integration layer

Layer 2 implements nine patterns from the Hohpe & Woolf EIP catalogue, under the book's own names. There is no framework to learn, no DSL and no XML: each component is a declaration you register in a catalogue and compose into pipelines. Conditions and mappings are written in FEEL, the same expression language your decision tables already use.

Message Channel

A named conduit with two delivery modes: point-to-point, where each message reaches exactly one consumer, or publish-subscribe, where each message reaches every subscriber. A channel is typed to a declared item definition, so the schema a channel carries is part of the topology rather than an assumption each consumer makes privately.

Message Endpoint

Connects an external service to a channel, referencing the service by its identity in your ArchiMate model rather than by a hostname. Inbound endpoints are entry points; messages queue there until the engine consumes them. Because the reference is to the architecture model, "which services can put a message on this channel?" is a question your model answers.

Content-Based Router

Routes using FEEL conditions evaluated against the message payload. Routes are prioritised, the first matching route wins, and an empty condition is the catch-all default. Exactly one branch fires per message.

router  claim-router

  priority 10   →  channel high-value      when   amount > 10000
  priority  5   →  channel auto-approve    when   amount <= 500
  default       →  channel standard

Message Filter

Drops messages that do not satisfy a FEEL predicate. Non-matching messages are discarded and never reach the engine - useful for idempotency tokens (drop already-seen correlation keys) and for schema validation (drop malformed payloads before they raise incidents deep inside a process).

Aggregator

The most useful pattern for idempotency and fan-in. The Aggregator groups related messages by a correlation expression, holds them until a completion condition is satisfied or a timeout fires, and then releases the batch as a single message.

aggregator  order-results

  group by      orderId                 group messages sharing this key
  release when  count(items) >= 3       the completion condition
  or after      30s                     force-release regardless
  emit to       channel order-complete

In a worker-failure scenario: two workers both complete the same job before the lock timeout expires - a race. Both completions arrive at the Aggregator under the same correlation key, and the completion condition expects exactly one result, so the second is held and never forwarded. The engine sees one completion per job, server-side, with no worker-side logic required.

Correlation Identifier

Holds the mapping from a correlation key to the process instance waiting on it. The canonical use case is a BPMN receive task: the task suspends the instance and registers the key it is waiting for. When a message arrives with a matching key, the match returns that entry and removes it in the same atomic step, so a second message with the same key matches nothing. That is first-match-wins deduplication enforced at the engine boundary rather than in every worker that might send a duplicate.

correlation identifier   key expression: paymentId

  receive task activates   →  register  paymentId  →  instance, resume point
  confirmation arrives     →  match     paymentId  →  entry found, removed
                                                       instance resumes
  duplicate arrives        →  match     paymentId  →  nothing; ignored

Splitter

Produces many output messages from one input: a FEEL expression evaluates to a list and one message is emitted per element. Use it for fan-out - a single order confirmation that needs to trigger a fulfilment task, a notification task, and an audit task in parallel, each as an independent process instance.

Message Translator

Declares a schema transformation between two item definitions as a FEEL mapping expression, which separates integration concerns - field renaming, unit conversion, restructuring - from process logic. The process receives a correctly shaped payload regardless of what the upstream service emitted, and the mapping is a declaration you can inspect rather than code buried in an adapter.

Pipes and Filters

An ordered chain of the components above. A message enters at the first step and passes through each in order - a typical chain being Filter, then Translator, then Router, then Aggregator - with the output of one feeding the input of the next. If a filter rejects the message, execution stops there and nothing downstream sees it.

pipeline  payment-ingest

  step 1   filter       duplicate-filter
  step 2   translator   schema-translator
  step 3   router       value-router
  step 4   aggregator   result-aggregator

4. Idempotency without a broker

The typical argument against REST polling for job workers is that it requires idempotency to be implemented by each worker individually. If a worker dies mid-execution, the job re-enters the queue and a second worker picks it up. Without a deduplication mechanism, the same logical action executes twice.

Layer 2 eliminates this requirement on the worker side. Here is the full server-side idempotency stack:

The critical insight: none of these mechanisms require the worker to maintain state. Workers remain stateless. The engine is the deduplication fence.

5. When to use which layer

The decision is not permanent. You start with Layer 1 and add integration components incrementally when a specific requirement surfaces. There is no migration, no data re-modelling, and no service to deploy. You declare a pipeline or an aggregator, the engine picks it up, and the processes already running are unaffected.

6. The scale story

REST polling has a real ceiling. At tens of thousands of concurrent workers with sub-second polling intervals, the activate endpoint becomes a bottleneck. This is acknowledged honestly: for extremely high-frequency job throughput, a message queue (Kafka, RabbitMQ, SQS) in front of the activate endpoint is the right call.

But that ceiling is much higher than most assume, and the integration layer raises it further. Consider:

• A worker polling every second with an average job duration of 10 seconds generates approximately one activate request every 10 seconds while busy.
• An Aggregator batching completions cuts round-trips for high-frequency short jobs.
• A Message Filter drops invalid or duplicate completions before they reach the engine, reducing contention on the instance lock.

For the use cases Priostack is designed for - BPM consultant tooling, enterprise approval flows, architecture-to-execution workflows - the ceiling is never reached. The typical deployment has dozens to hundreds of concurrent instances, not millions. You add a broker when you have a broker-scale problem. Not before.

If you want to see the integration layer in action, the EIP Pipelines article walks through two complete topologies - a big-data corpus-building pipeline and a fast-data anomaly-routing pipeline - built entirely with these nine primitives.