Insights Contact

Data, AI and Testing

How Kafka Powers Payments: The Few Seconds After You Tap Your Card

By SP Sankar

When you tap your card for a $40 coffee, it feels like one simple step. Behind the scenes, many teams must act on that one payment within seconds. Many payment companies use Apache Kafka to make that happen.

One tap, many jobs

  • Fraud checks that it’s really you.
  • The ledger records the money exactly once.
  • Alerts send you a text or app notification.
  • AML monitoring watches for money laundering.
  • Settlement makes sure the shop gets paid.

How Kafka helps

Diagram: payment, refund and dispute systems write events once to Kafka topics; fraud check, ledger, customer alert, AML monitoring and settlement teams each read what they need
Payments, refunds and disputes flowing through one Kafka

Read from left to right. On the left are the producers: card payment, refund and dispute systems, each writing every event once. In the middle, Kafka keeps each kind of event on its own topic. On the right are the consumers, like fraud, ledger, alerts, AML and settlement, each reading what it needs. The bar at the bottom sums it up: written once, read by many.

Kafka termIn plain English
ProducersThe systems that create events: payments, refunds and disputes. Each writes an event once.
TopicsLabeled shelves inside Kafka: a “payments” shelf, a “refunds” shelf, a “disputes” shelf. Events stay in order and are kept for a set time.
ConsumersThe teams that read from the shelves, each at its own pace, taking only what they need.

The payment system writes the payment once, to the “payments” topic, and each team reads it from there. If a new team arrives tomorrow, it simply starts reading. Nobody has to change the payment system.

The “Approved” you see on the card machine comes straight back from the payment system. It doesn’t wait for Kafka.

Why payments teams trust it

PromiseHow Kafka delivers it
No lost paymentsEvery message is copied to several servers, so one server failing loses nothing
Right orderEvents for the same card or account can be kept in order, so a refund never appears before the purchase
No double savesA setting called “idempotent producer” (safe to repeat) stops the same event being written twice when the network retries
Replay for auditsMessages are kept for a set time, so teams can re-read past payments to fix errors, check that records match (reconcile) or answer an auditor
Real timeFraud and AML checks run while the payment happens, not the next morning

My take

Kafka lets one payment event feed every team that needs it, quickly, in order and without loss. Write once, read by many. It isn’t magic, though: I’ve seen Kafka queues back up on a high-volume day. I share what happened, and how we fixed it, in my post on peak payment days.

Working on a payments project?

I help banks and fintechs deliver real-time payments, ISO 20022, and AI-driven AML and compliance change, from strategy to go-live.

Connect on LinkedInSee how I can help