Scale Yazılım Training & Consulting · Kafka, Spark & Big Data
Training & consulting · Data platforms

Data pipelines that keep up with the stream.

Consulting and training for Apache Kafka, Apache Spark and the wider big data stack — Hadoop, Hive and Trino. Since 2014 we have designed streaming and batch pipelines, tuned the jobs that are slow and expensive, and trained the engineers who run them.

Fig. 01 · Topic & consumers
Click — or press space — to send a burst of records A Kafka topic with three partitions, read by Spark executors. New records enter on the left; glowing records are not yet read — that is consumer lag. Read records stay in the log until retention drops them.
Made with ISO-GLOW ↗
  • 2014Building data platforms for data-heavy teams
  • 4Programmes: Kafka, Spark, the Hadoop ecosystem, PySpark
  • ≤ 15People per class, with a cloud lab for each of them
  • EN · TRDelivered in English or Turkish, remote or on-site
Consulting

Make the pipeline boring. Topic design, job tuning and table layouts that keep latency low and costs predictable.

Training

From first topic to platform. Kafka, Spark, Hive, Trino and PySpark, taught with labs on realistic data volumes.

Support

Stay on top of the load. Capacity reviews, upgrades such as ZooKeeper to KRaft, and help when lag will not drain.

Scale Yazılım Ltd. Şti. · Ankara, Türkiye scaleyazilim.com/services/big-data · info@scaleyazilim.com · 1 / 3
Scale Yazılım Training & Consulting · Kafka, Spark & Big Data
02 · Consulting

Streams that drain, jobs that finish.

Data platforms slow down in predictable places: partitions chosen years ago, skewed joins, small files, lag nobody watches. We find the bottleneck with your metrics, fix it with your team and leave the monitoring that shows it stays fixed.

01

Streaming architecture

Topic and partition design, keys and ordering, retention and compaction, schema registry, exactly-once.

You getA reviewed event model and a capacity plan.

02

Kafka operations

KRaft migration, broker sizing, rack awareness, replication and min ISR, rebalancing storms, MirrorMaker.

You getTuned clusters and lag alerts that mean something.

03

Spark performance

Skew, shuffles, broadcast joins and salting, partitioning, AQE, executor memory, small files.

You getFaster, cheaper jobs with before/after run times.

04

Lakehouse & warehouse

Parquet, Iceberg and Delta tables, partitioning, the Hive metastore and Trino for interactive SQL.

You getA table layout that queries fast and costs less.

05

Integration & CDC

Kafka Connect, Debezium change data capture, sinks into search, vector and warehouse stores; RabbitMQ bridges.

You getPipelines with replay, dead-letter queues and monitoring.

06

Platform on Kubernetes

Strimzi for Kafka, the Spark operator and Trino on Kubernetes; autoscaling and cost control.

You getA reproducible platform defined as code.

Signals it is time to call

  • Consumer lag grows every evening and drains by morning
  • One Spark task runs for an hour while the rest take minutes
  • The nightly batch no longer finishes before the workday
  • Partition counts were picked years ago and never revisited
  • Millions of small files slow every query
  • ZooKeeper to KRaft is on the roadmap but nobody owns it

How an engagement runs

  1. Scoping call 45 min · free

    We hear the problem, agree goals and the numbers success is judged by.

  2. Assessment 1–2 weeks

    Read-only access, diagnostics, logs and short interviews. Findings in writing.

  3. Fix & tune 2–6 weeks

    Changes made with your team in small steps, each measured before and after.

  4. Hand-over ongoing

    Runbooks, a focused workshop for your engineers and optional retained support.

Field experience Banking & finance Telecom Public sector Defence Retail & e-commerce Logistics
Scale Yazılım Ltd. Şti. · Ankara, Türkiye scaleyazilim.com/services/big-data · info@scaleyazilim.com · 2 / 3
Scale Yazılım Training & Consulting · Kafka, Spark & Big Data
03 · Training

Four programmes, one platform.

Take them on their own or as a path. Each programme runs on a cloud lab with realistic data volumes and ends with a project built by the participants.

Programme 01 · Streaming

Apache Kafka

5 days · 30 hours
Level
Beginner – intermediate
Prereq.
Basic Linux
  • Architecture and installation: brokers, ZooKeeper and KRaft
  • Partitions, replication, topic design and retention
  • Kafka Connect: sources, sinks and connector operations
  • Streams API, REST proxy and custom serializers
  • Kafka and RabbitMQ: when to use which, and bridging them
Programme 02 · Processing

Apache Spark

4 days · 24 hours
Level
Intermediate – advanced
Prereq.
Linux and containers
  • RDDs, DataFrames and Spark SQL; ETL jobs
  • Performance: joins, broadcasting, salting, cluster management
  • Structured Streaming from Kafka and files, windowing
  • Reporting with Grafana and Kibana
  • Project day: an analysis on real data, built in teams
Programme 03 · Platform

Big Data Ecosystem on Kubernetes

5 days · 30 hours
Level
Intermediate – advanced
Prereq.
Linux and containers
  • Big data concepts, batch and stream; HDFS architecture
  • Hadoop, HDFS file operations and MapReduce basics
  • Hive: architecture, DDL and DML, performance
  • Real-time ingestion with Kafka
  • Spark, Kafka and Trino on Kubernetes, managed with operators
Programme 04 · Analysis

Python for Data Analysis & PySpark

5 days · 30 hours
Level
Beginner – intermediate
Prereq.
Basic programming
  • Python fundamentals, environments and Jupyter
  • SQL, SQLite and SQLAlchemy; Pandas with SQL
  • NumPy arrays, broadcasting and linear algebra
  • Pandas: selection, cleaning, groupby, joins, pivots, time series
  • PySpark: DataFrames, transformations and actions, Spark SQL
  • DeliveryRemote or on-site; daytime, evening or weekend sessions
  • LabsA cloud lab for every participant, on realistic data
  • Class sizeUp to 15 people, so everyone gets hands-on time
  • LanguageEnglish or Turkish, with all materials provided
  • TailoringContent, length and examples fitted to your stack
Next step

Start with a 45-minute scoping call.

Tell us what is slow, wrong or about to grow. We reply within one working day with a suggested next step — consulting, training or both.

Scale Yazılım Ltd. Şti. · Ankara, Türkiye scaleyazilim.com/services/big-data · info@scaleyazilim.com · 3 / 3