⚡ archetype: streaming

Process events at p99 < 1s without the duplicate-charge incident.

Building with Kafka, Kinesis, Pulsar, Flink, Beam, or Debezium CDC? GreatCTO auto-detects the streaming archetype and ships exactly-once semantics, idempotency proofs, backpressure strategy, DLQ + poison-message handling, schema evolution, and checkpoint storage gates from day one.

$ npx great-cto init Back to home ↗

What you avoid

The 5 streaming bugs that page you at 4am.

Without GreatCTO

☐At-least-once consumer + non-idempotent sink → duplicate $400 charges
☐No DLQ — poison message blocks topic for 6 hours
☐Schema change deployed without registry — every consumer dies
☐Backpressure unhandled — Kafka lag 2 days · users see stale data
☐Stateful Flink job crashes — no checkpoint, lost state, manual replay
☐One bad event · all-hands incident · trust gone.

With GreatCTO

✓streaming-reviewer enforces exactly-once on payment paths
✓DLQ topic per consumer · retry+move policy · alert on rate spike
✓Schema Registry with backward-compat enforced · breaking change procedure
✓Lag alerting at 60s/5min · capacity tested in staging · replay tooling
✓Checkpoint storage configured · savepoints before deploy · TTL on state
✓p99 < 1s · zero duplicate charges · zero data loss.

Auto-applied gates

Detected: `kafkajs` / `@confluentinc/kafka-javascript` / `debezium` →
streaming archetype.

Compliance auto-suggested: gdpr · soc2-cc7. Specialist agents activated:

01 · streaming-reviewer

Delivery + ordering + DLQ

At-most-once / at-least-once / exactly-once decision · idempotency proof · partition-key strategy · backpressure mechanism · DLQ + poison-message handling · schema evolution · stateful checkpoint storage.

02 · performance-engineer

p99 latency budget

End-to-end p50 / p95 / p99 targets · tail-latency causes (GC, network, checkpoint pauses) · throughput capacity test · load test with realistic burst.

03 · data-platform-reviewer

Downstream warehouse load

Stream → batch handoff · GDPR retention in event store · CDC fidelity (snapshot + log-based replication) · OpenLineage event emission.

04 · security-officer

Event PII + audit

PII classification per event type · authorization on producer/consumer · audit log emission for regulated topics · encryption in flight (TLS) + at rest.

Domain pack overlays

Likely to overlay on streaming.

Packs auto-attach when CLI detects pack-specific signals (e.g. twilio in deps → voice-pack). Each pack adds its own reviewer agents + human gates on top of the base archetype pipeline.

+ API Platform

OAuth 2.1, webhook signing, idempotency, RFC 8594 Sunset