Insights Contact

Data, AI and Testing

Risk-Based Testing for Payments: Test What Can Hurt Most, First

By SP Sankar

No payments release ever has enough time to test everything. New rails, new message formats, fee changes, a vendor upgrade and a regulator deadline all land in the same few weeks. Risk-based testing is the simple answer: find what can hurt most, test it first and test it hardest, and be honest about what you tested lightly.

The idea is not new. The ISTQB glossary and the ISO/IEC/IEEE 29119 testing standard both describe risk as the way to decide what to test, how deeply, and in what order. This guide shows how to do it on a real payments project, with a scoring method, a worked example and a checklist. For ready-made test cases, see our Testing Playbook — Payments.

Why payments need it

  • Money moves once. A duplicate wire or a payment sent to the wrong account is hard to pull back. A screen with the wrong colour is not.
  • Deadlines don’t move. Cut-off times, payroll dates and scheme mandate dates are fixed, so test time gets squeezed, not the go-live date.
  • Regulators and card schemes expect it. Bank supervisors expect controls that match the size of the risk, and they ask how you decided what was tested before go-live.
  • Not everything is equal. A rule change to sanctions screening and a change to a help-page link are both “changes”, but only one can cause a fine.

How to score a risk

Every risk gets two scores from 1 (low) to 5 (high). Multiply them to get the risk score, from 1 to 25.

Likelihood: how likely is a defect?Impact: how bad if it reaches production?
New or heavily changed codeMoney lost, duplicated or sent to the wrong party
Complex logic: fees, FX, cut-offs, holidaysNumber of customers or payments affected
Many systems or vendors in the flowBreach of a regulation or scheme rule (sanctions, Nacha, card rules)
A history of defects or incidents in this areaA missed cut-off or settlement deadline
New team, new vendor or new technologyHarm to customers’ trust or the bank’s name
Manual steps or weak test dataHow hard it is to spot and fix in production

One rule on top: any risk with impact 5 for money or regulation is treated as high, even if its likelihood is low. A rare defect in sanctions screening is still a defect you cannot afford.

Impact →Likelihood →MediumHighHighLowMediumHighLowLowMediumLowMediumHigh
Top right is tested first and deepest. Bottom left gets a light touch.

How deep to test at each level

Risk levelScoreWhat testing looks likeWho signs off
High15 to 25, or any impact-5 money or regulatory riskEvery path, negative and edge cases, cut-offs and holidays, retries and timeouts, performance at peak volume, added to automated regression, reviewed by a second testerBusiness owner and, where needed, compliance
Medium8 to 14Main paths plus the most likely negative cases, key data combinations, regression of nearby featuresQA lead and product owner
Low1 to 7Happy path and a short exploratory session; fix defects if found, but don’t hold the release for cosmetic onesQA lead

A worked example

A bank’s next release adds outgoing FedNow payments, changes how fees are shown, and updates the app’s logo. The team lists the risks together and scores them:

RiskLikelihoodImpactScoreLevel
A timeout and retry sends the same instant payment twice4520High
Payment status stays “pending” after the network times out3515High
Sanctions screening is skipped for the new rail2510High (impact-5 rule)
Wrong fee shown before the customer confirms339Medium
New logo looks blurred on small phones212Low

The test plan now writes itself. Most of the effort goes to duplicate checks, timeouts and screening. The fee screen gets solid coverage. The logo gets a quick look on a few devices.

Six steps that work

  1. List the risks with the right people. Product, payment operations, compliance, developers and QA in one short workshop. Each group sees risks the others miss. Start from the requirements, past incidents and the RAID log.
  2. Score them. Use the likelihood and impact factors above. Agree the scores as a group, not alone in QA.
  3. Link each risk to tests. Give each risk an ID and tag the test cases that cover it. Any high risk with no test is a gap you can see at once.
  4. Run tests in risk order. High risks first, so that if time runs out, what is left untested is the least harmful.
  5. Report by risk, not by test count. “95% of tests passed” says little. “All high risks tested and passed; two medium risks open, with workarounds” is what a go/no-go meeting needs.
  6. Score again every release. Update the list after each production incident, defect trend or big change. Risks move.

Don’t forget the risks you can’t see in a test case

Functional tests miss some of the biggest payment risks: peak volume, slow queues, failover between data centres, and expired certificates. Score these too. Busy days such as payday and month-end find weak spots that quiet test days never show; see how banks handle peak payment days.

Common mistakes

  • Everything is high. If every risk is high, nothing is. Keep high for what can truly cause loss, harm or a breach.
  • QA scores alone. Testers know where bugs hide, but operations and compliance know what hurts. Score together.
  • Low means untested. Low risk means a lighter test, not none.
  • The list never changes. A risk list from the first sprint is out of date by the last.
  • No record of decisions. Auditors will ask why something was tested lightly. Keep the scores and who agreed them.

Who owns what

  • QA or test lead: runs the workshop, keeps the risk list and links risks to tests.
  • Product owner: agrees impact scores and accepts any risk left open at go-live.
  • Developers and architects: judge likelihood, especially for new code and integrations.
  • Payment operations: brings past incidents and what goes wrong on busy days.
  • Compliance: flags regulatory and scheme risks and signs off on them.

A quick checklist

  • A risk list exists for the release, scored for likelihood and impact.
  • Every high risk has linked test cases, and they have run.
  • Money and regulatory risks with impact 5 are treated as high.
  • Non-functional risks (peak load, failover, certificates) are on the list.
  • The test report shows status by risk, including risks still open.
  • Open risks at go-live are accepted in writing by the business owner.
  • The list is updated after every production incident.

My take

Risk-based testing is not about testing less. It is about spending the time you have where the money is. The teams that do it well don’t have more testers; they have one shared risk list that product, operations and compliance agreed to, and they can say in one sentence what was tested hard and why. Pair this method with the scenarios in our Testing Playbook — Payments and you have both the “what to test first” and the “how”.

Working on a payments project?

I help banks and fintechs deliver real-time payments, ISO 20022, and AI-driven AML and compliance change, from strategy to go-live.

Connect on LinkedInSee how I can help