No payments release ever has enough time to test everything. New rails, new message formats, fee changes, a vendor upgrade and a regulator deadline all land in the same few weeks. Risk-based testing is the simple answer: find what can hurt most, test it first and test it hardest, and be honest about what you tested lightly.
The idea is not new. The ISTQB glossary and the ISO/IEC/IEEE 29119 testing standard both describe risk as the way to decide what to test, how deeply, and in what order. This guide shows how to do it on a real payments project, with a scoring method, a worked example and a checklist. For ready-made test cases, see our Testing Playbook — Payments.
Why payments need it
- Money moves once. A duplicate wire or a payment sent to the wrong account is hard to pull back. A screen with the wrong colour is not.
- Deadlines don’t move. Cut-off times, payroll dates and scheme mandate dates are fixed, so test time gets squeezed, not the go-live date.
- Regulators and card schemes expect it. Bank supervisors expect controls that match the size of the risk, and they ask how you decided what was tested before go-live.
- Not everything is equal. A rule change to sanctions screening and a change to a help-page link are both “changes”, but only one can cause a fine.
How to score a risk
Every risk gets two scores from 1 (low) to 5 (high). Multiply them to get the risk score, from 1 to 25.
| Likelihood: how likely is a defect? | Impact: how bad if it reaches production? |
|---|---|
| New or heavily changed code | Money lost, duplicated or sent to the wrong party |
| Complex logic: fees, FX, cut-offs, holidays | Number of customers or payments affected |
| Many systems or vendors in the flow | Breach of a regulation or scheme rule (sanctions, Nacha, card rules) |
| A history of defects or incidents in this area | A missed cut-off or settlement deadline |
| New team, new vendor or new technology | Harm to customers’ trust or the bank’s name |
| Manual steps or weak test data | How hard it is to spot and fix in production |
One rule on top: any risk with impact 5 for money or regulation is treated as high, even if its likelihood is low. A rare defect in sanctions screening is still a defect you cannot afford.
How deep to test at each level
| Risk level | Score | What testing looks like | Who signs off |
|---|---|---|---|
| High | 15 to 25, or any impact-5 money or regulatory risk | Every path, negative and edge cases, cut-offs and holidays, retries and timeouts, performance at peak volume, added to automated regression, reviewed by a second tester | Business owner and, where needed, compliance |
| Medium | 8 to 14 | Main paths plus the most likely negative cases, key data combinations, regression of nearby features | QA lead and product owner |
| Low | 1 to 7 | Happy path and a short exploratory session; fix defects if found, but don’t hold the release for cosmetic ones | QA lead |
A worked example
A bank’s next release adds outgoing FedNow payments, changes how fees are shown, and updates the app’s logo. The team lists the risks together and scores them:
| Risk | Likelihood | Impact | Score | Level |
|---|---|---|---|---|
| A timeout and retry sends the same instant payment twice | 4 | 5 | 20 | High |
| Payment status stays “pending” after the network times out | 3 | 5 | 15 | High |
| Sanctions screening is skipped for the new rail | 2 | 5 | 10 | High (impact-5 rule) |
| Wrong fee shown before the customer confirms | 3 | 3 | 9 | Medium |
| New logo looks blurred on small phones | 2 | 1 | 2 | Low |
The test plan now writes itself. Most of the effort goes to duplicate checks, timeouts and screening. The fee screen gets solid coverage. The logo gets a quick look on a few devices.
Six steps that work
- List the risks with the right people. Product, payment operations, compliance, developers and QA in one short workshop. Each group sees risks the others miss. Start from the requirements, past incidents and the RAID log.
- Score them. Use the likelihood and impact factors above. Agree the scores as a group, not alone in QA.
- Link each risk to tests. Give each risk an ID and tag the test cases that cover it. Any high risk with no test is a gap you can see at once.
- Run tests in risk order. High risks first, so that if time runs out, what is left untested is the least harmful.
- Report by risk, not by test count. “95% of tests passed” says little. “All high risks tested and passed; two medium risks open, with workarounds” is what a go/no-go meeting needs.
- Score again every release. Update the list after each production incident, defect trend or big change. Risks move.
Don’t forget the risks you can’t see in a test case
Functional tests miss some of the biggest payment risks: peak volume, slow queues, failover between data centres, and expired certificates. Score these too. Busy days such as payday and month-end find weak spots that quiet test days never show; see how banks handle peak payment days.
Common mistakes
- Everything is high. If every risk is high, nothing is. Keep high for what can truly cause loss, harm or a breach.
- QA scores alone. Testers know where bugs hide, but operations and compliance know what hurts. Score together.
- Low means untested. Low risk means a lighter test, not none.
- The list never changes. A risk list from the first sprint is out of date by the last.
- No record of decisions. Auditors will ask why something was tested lightly. Keep the scores and who agreed them.
Who owns what
- QA or test lead: runs the workshop, keeps the risk list and links risks to tests.
- Product owner: agrees impact scores and accepts any risk left open at go-live.
- Developers and architects: judge likelihood, especially for new code and integrations.
- Payment operations: brings past incidents and what goes wrong on busy days.
- Compliance: flags regulatory and scheme risks and signs off on them.
A quick checklist
- A risk list exists for the release, scored for likelihood and impact.
- Every high risk has linked test cases, and they have run.
- Money and regulatory risks with impact 5 are treated as high.
- Non-functional risks (peak load, failover, certificates) are on the list.
- The test report shows status by risk, including risks still open.
- Open risks at go-live are accepted in writing by the business owner.
- The list is updated after every production incident.
My take
Risk-based testing is not about testing less. It is about spending the time you have where the money is. The teams that do it well don’t have more testers; they have one shared risk list that product, operations and compliance agreed to, and they can say in one sentence what was tested hard and why. Pair this method with the scenarios in our Testing Playbook — Payments and you have both the “what to test first” and the “how”.
Working on a payments project?
I help banks and fintechs deliver real-time payments, ISO 20022, and AI-driven AML and compliance change, from strategy to go-live.
Connect on LinkedInSee how I can help