Streaming architecture
Topic and partition design, keys and ordering, retention and compaction, schema registry, exactly-once.
You getA reviewed event model and a capacity plan.
Consulting and training for Apache Kafka, Apache Spark and the wider big data stack — Hadoop, Hive and Trino. Since 2014 we have designed streaming and batch pipelines, tuned the jobs that are slow and expensive, and trained the engineers who run them.
Make the pipeline boring. Topic design, job tuning and table layouts that keep latency low and costs predictable.
From first topic to platform. Kafka, Spark, Hive, Trino and PySpark, taught with labs on realistic data volumes.
Stay on top of the load. Capacity reviews, upgrades such as ZooKeeper to KRaft, and help when lag will not drain.
Data platforms slow down in predictable places: partitions chosen years ago, skewed joins, small files, lag nobody watches. We find the bottleneck with your metrics, fix it with your team and leave the monitoring that shows it stays fixed.
Topic and partition design, keys and ordering, retention and compaction, schema registry, exactly-once.
You getA reviewed event model and a capacity plan.
KRaft migration, broker sizing, rack awareness, replication and min ISR, rebalancing storms, MirrorMaker.
You getTuned clusters and lag alerts that mean something.
Skew, shuffles, broadcast joins and salting, partitioning, AQE, executor memory, small files.
You getFaster, cheaper jobs with before/after run times.
Parquet, Iceberg and Delta tables, partitioning, the Hive metastore and Trino for interactive SQL.
You getA table layout that queries fast and costs less.
Kafka Connect, Debezium change data capture, sinks into search, vector and warehouse stores; RabbitMQ bridges.
You getPipelines with replay, dead-letter queues and monitoring.
Strimzi for Kafka, the Spark operator and Trino on Kubernetes; autoscaling and cost control.
You getA reproducible platform defined as code.
We hear the problem, agree goals and the numbers success is judged by.
Read-only access, diagnostics, logs and short interviews. Findings in writing.
Changes made with your team in small steps, each measured before and after.
Runbooks, a focused workshop for your engineers and optional retained support.
Take them on their own or as a path. Each programme runs on a cloud lab with realistic data volumes and ends with a project built by the participants.
Start with a 45-minute scoping call.
Tell us what is slow, wrong or about to grow. We reply within one working day with a suggested next step — consulting, training or both.