Insights Contact

Data, AI and Testing

Sell Me This Defect: How to Get Bugs Fixed Faster

By SP Sankar

⏱ 3-minute read · open “Read more” for the details

Why do some bugs, even critical ones, sit open for months? In my experience it’s rarely the code. It’s how the bug is “sold”. It isn’t explained clearly, the urgency isn’t clear, it’s ranked wrongly, or the right people never see it. A good defect report is a short sales pitch: what’s broken, who it hurts, and why it matters now.

A bug gets fixed faster when you ‘sell’ it well: show clearly what is broken, who it hurts and how much it costs. This post shows how to write a defect that people act on.

The bug life cycle

Meet Maya, a tester. She finds a bug: a refund stays “Pending” even after it succeeds. Like every bug, it now has a life cycle.

  • New, then assigned, fixed, retested and closed.
  • If the retest fails, it goes back to its owner.
  • Some bugs wait for a later release.
  • Tip: a short daily triage meeting finds the right owner fast.
The bug life cycleThe main path runs down; side paths show the other ways a bug can go1OpenBug logged2AssignedTriage picks an owner3In progressFix being built4FixedDeployed to test5RetestTester checks the fix6ClosedConfirmed fixedRejectedNot a bug, a duplicate, orworks as designed. Ends here.DeferredValid, but moved to alater releasewhen plannedRetest fails: reopenedBug 1: passes first timeBug 2: retest fails onceBug 3: deferred, then fixedPassed first time! ✓Retest failed!Bug squashed! ✓Moved to a later releaseFixed in a later release ✓
Read more: The bug life cycle

Watch the three bugs. Bug 1 passes its retest the first time and goes straight to Closed, like most bugs. Bug 2 fails its retest, goes back to its owner, and closes on the second try. Bug 3 is deferred: it waits for a later release, comes back to Assigned when that release is planned, and is then fixed and closed. Follow the six boxes from top to bottom: opened, assigned, fixed, deployed for testing, retested and closed. The dashed line on the left: if the retest fails, the bug is reopened and goes back to its owner. On the right are the two other ways out of triage. Rejected means it’s not a real bug, so it ends there (a bug that belongs to another team is reassigned instead). Deferred means it’s valid but moved to a later release.

StatusWhat it meansUsually moved by
OpenA bug is found and loggedTester, user, product owner, developer or customer
AssignedTriage gives it to the right team; unclear bugs go to the product owner or BA. A bug that belongs to another team is reassigned to them, not rejected.Test lead, Scrum Master, product owner or dev manager
In progressThe fix is being builtDeveloper
Fixed / RetestThe fix is deployed to a test environment and ready to checkDev lead or release manager
ReopenedThe retest failedTester
RejectedNot a real bug: a duplicate, a test data error, or the system works as designedTest lead, Scrum Master or product owner
DeferredValid, but moved to a later release or waiting on a dependencyProduct owner
ClosedRetested and confirmed fixedTester

Tip: when many systems are involved, hold a short daily triage meeting (sorting new bugs by how serious they are). Finding the right owner quickly is half the fix.

How to “sell” a defect

Maya’s bug gets fixed fast only if she “sells” it well.

  • A clear title: what is broken, and where.
  • Steps to reproduce, with test data and the environment.
  • Expected vs actual result, with a screenshot or log.
  • Who it hurts and what it costs.
Read more: How to “sell” a defect
  • A clear title: what is broken, and where. For example, “Refund stays Pending after success: EUR cards, iOS app”.
  • Business impact: how many customers, how much money, any compliance risk.
  • Severity and priority: severity is how bad it is; priority is how soon it must be fixed.
  • Root cause category once known, so trends can be fixed at the source.

Severity vs priority

Next, the team decides how bad it is and how soon to fix it.

  • Severity: how badly it breaks the system. The tester sets it.
  • Priority: how soon it must be fixed. The product owner sets it.
  • They usually match, but not always.
Read more: Severity vs priority

Severity is how badly the bug breaks the system. Priority is how soon it must be fixed. The tester usually sets severity. The product owner sets priority, based on business impact. They often match, but not always.

SeverityWhat it meansPayments example
Critical (S1)Money is wrong, lost or stuck, or a required check doesn’t run. No workaround.Payments post twice; sanctions screening is skipped
High (S2)A main feature fails, but there is a workaroundWires can’t be approved in the app, but the bank’s wire operations manager can approve them manually
Medium (S3)A feature only partly worksThe payment history filter shows the wrong dates
Low (S4)Cosmetic or minorA typo on the receipt screen

When severity and priority don’t match:

SeverityHigh priorityLow priority
High severityFix now. Duplicate payments in production.Serious, but not urgent yet. Payments to a new country fail every time, but that country only goes live next quarter. No customer is affected today, so it is planned, not rushed.
Low severitySmall, but urgent. The wrong bank name or logo on the login page on launch day. Nothing breaks, but every customer sees it.Fix when convenient. A misaligned icon on an internal admin screen.
  • Agree the definitions, with examples, before the project starts. Then triage is a quick decision, not a debate.
  • Priority can change. A low-priority bug can jump the queue before a peak payment day or an audit.
  • In payments, start high. Anything that touches money, compliance or customer data is treated as high severity until proven otherwise.

Common defect categories

Tagging the type of bug helps the right team pick it up.

  • Common types include functional, data, interface, screen, performance and security bugs.
  • Tag every bug, so patterns show up in the reports.
Read more: Common defect categories
CategoryExamples
RequirementMissing or unclear requirements. Usually fewer than functional bugs, but costly, because they are often found late.
FunctionalA button doesn’t respond; a feature gives the wrong result
DataBad test data, migration errors, invalid set-up
EnvironmentServer or database down, wrong environment, expired tokens
InterfaceA partner API fails or returns errors like 502, 503 or 504
PerformanceSlow pages, timeouts, problems under many users
SecurityWeak authentication, exposed data, cross-site scripting
AccessibilityPeople with disabilities can’t use it (for example, Section 508 or WCAG gaps: missing the rules that make websites usable for people with disabilities)
Usability and contentConfusing screens, unclear messages, broken images or videos
CompatibilityWorks on one browser, device or OS but not another
Not reproducibleHappens only sometimes, such as network-related crashes
Tester errorNot a real bug: a test set-up or understanding issue

Interface bugs: upstream, downstream, external and legacy

Maya’s refund passes through several systems. Many bugs live in the connections between them.

  • Tag every interface bug with the connection that broke.
  • Agree an interface contract that both sides test against.
  • If the other side isn’t ready, test with a stub, then retest for real.
Interface bugs by connectionThe 15 interface bugs, split by which connection brokeDownstreamDownstream: 5 interface bugs5UpstreamUpstream: 4 interface bugs4ExternalExternal: 4 interface bugs4LegacyLegacy: 2 interface bugs2Downstream breaks most: 5 of 15Example data
Read more: Interface bugs: upstream, downstream, external and legacy

A payment passes through many systems, and many bugs live in the connections between them. Tag every interface bug with which connection broke, so the right team owns it.

ConnectionWhat it meansTypical bugs
UpstreamSystems that send data to us: the mobile app, online banking, a corporate client’s payment fileMissing or wrong fields, a new file format nobody was told about, the same file sent twice
DownstreamSystems we send data to: core banking, the ledger, statements, finance reportsPayment sent but not posted, totals that don’t match, a field cut short
ExternalSystems outside the bank: Fedwire, ACH, card networks, sanctions screening vendors, partner APIsTimeouts, error replies (502, 503, 504), an expired certificate, a network rule change
LegacyOld core systems, often mainframes with nightly batch jobs and fixed-width filesFields cut to a fixed length, leading zeros dropped, payments waiting for the night batch
  • Agree an interface contract (the exact format, fields and error codes both sides follow), and test against it.
  • When the other side isn’t ready, test with a stub (a stand-in that answers like the real system), then retest with the real system in SIT.
  • Book external test environments early. Payment networks have their own test and certification windows.
  • For legacy systems, test the batch window: what happens to a payment that arrives while the night batch is running?
  • Log the bug against the team that owns the broken side. When it’s unclear, link both teams to one bug instead of opening two.

Bugs by channel: UI, mobile, web, API and legacy

Customers meet bugs on a screen. Maya found hers in the iOS app.

  • Tag each bug with its channel: web, mobile, API or legacy.
  • Test on the devices and browsers customers really use.
  • Test the money moments: a double tap, the back button, the app closing, the network dropping.
Bugs by applicationWhere the 198 bugs showed upMobile appMobile app: 64 bugs64APIs and backendAPIs and backend: 57 bugs57WebWeb: 51 bugs51LegacyLegacy: 26 bugs26Mobile app: 64 of 198 bugs (32%)Example data
Read more: Bugs by channel: UI, mobile, web, API and legacy

Customers and staff meet bugs on a screen. Tag each bug with the application where it showed up, so you can see which channel needs more testing.

ChannelWhat it coversTypical bugs
UI (user interface: any screen)What the user sees and touches: layout, buttons, text and error messages, on mobile and webOverlapping text, a hidden button on small screens, an unclear error message, an amount shown without its currency
Mobile appThe iOS and Android appsWorks on one phone but not another, breaks after an OS update, the app closes in the middle of a payment, slow on a weak network
WebOnline banking in the browserWorks in Chrome but not Safari, a double click submits a payment twice, a session timeout loses the form, the back button resends
API (application programming interface: how systems and apps ask each other for data)The services behind the apps, and the APIs that corporate clients and partners call directlyWrong status code, a field renamed or missing, a change that breaks clients, a retry that pays twice, unclear error messages, an expired token
LegacyOld applications staff still use, such as mainframe “green screens” or desktop toolsField length limits, special characters rejected, data out of step with newer systems, and few people who know how they work
  • Test on the devices and browsers customers really use. Your analytics show the top ones.
  • Test the money-moving moments: a double tap, the back button, the app closing mid-payment, the network dropping. None of them may ever pay twice.
  • Test APIs directly, not only through the screens. Check the status code, every field and the error replies, and keep old API versions working for clients. See API automation with Rest-Assured.
  • Check accessibility on every new screen, not just before launch.
  • For legacy screens, test with the operations staff who use them every day. They know the shortcuts and the fields that matter.

Where bugs are caught: the test levels

Each test level checks something bigger than the one before.

  • Unit tests: developers check small pieces of code on every build.
  • System tests: testers check the whole application.
  • End-to-end and acceptance tests: the full journey, with business users.
Test levels, from small to realEach level catches bugs the one before cannot1Unit testingEach small piece works alone · Developers, on their own code2System testingThe whole application works · Testers, in a test environment3System integration testing (SIT)Systems talk to each other correctly · Testers, with other systems4End-to-end (E2E) testingA payment works from start to finish · Testers, the full journey5User acceptance testing (UAT)It does what the business needs · Business users6Penny testingA tiny real payment proves the live setup · Operations, in production
Read more: Where bugs are caught: the test levels

Each test level checks something bigger than the one before. A bug that slips through one level should be caught by the next.

  • Unit testing: developers test each small piece of code, automatically on every build. Example: the fee is rounded correctly.
  • System testing: testers check the whole application in a test environment. Example: create, approve and cancel a payment.
  • System integration testing (SIT): checks the connections to other systems, such as core banking, fraud screening and the payment network’s test system. Example: the payment message has the right format, and the reply is handled.
  • End-to-end (E2E) testing: one full customer journey across every system. Example: a customer sends a wire in the app, it is screened and sent, and it shows on the statement.
  • User acceptance testing (UAT): business users confirm, with real-life scenarios, that it does what they need before go-live.
  • Penny testing: in production, just before opening a new product to customers, or when connecting a new bank, rail or account, send a very small real payment, for example one cent or one dollar. Check that it arrives, posts and reconciles correctly. It is the only test that uses the real production setup: real certificates, real routing and real accounts. Agree it with operations and compliance first, use approved accounts, and return the money afterwards. Micro-deposits (two tiny amounts a customer confirms to prove a bank account is theirs) work on the same idea.

Bugs from automation, performance, security and compliance testing

Not every bug comes from a tester clicking through screens.

  • Automated tests catch repeat breaks fast.
  • Performance tests find slowdowns before peak days.
  • Security and compliance tests find what auditors would find.
Read more: Bugs from automation, performance, security and compliance testing

Not every bug comes from a tester clicking through screens. Tag every bug with its source, so you can see which kind of testing is paying off.

SourceWhat it findsWatch out for
Automation (tests that run by themselves on every change)Old features that broke after a new changeA failing test is not always a bug. Check for a flaky test or bad test data before you raise it.
Performance (load and stress tests)Slow screens, timeouts and failures at busy times, such as payday or Black FridayTest with real-life volumes and a production-like environment, or the results mislead
Security (code scans and penetration tests: experts trying to break in)Weak logins, exposed data and known flaws in code librariesLog these where only the right people can see them, and fix them by severity deadline
Compliance (checks against rules)Missed sanctions screening, wrong customer notices, wrong payment file formats, accessibility gapsThese often have a regulator’s deadline, so treat them as “must do” work

Story vs bug

Is Maya’s issue a bug or a new request? The label decides how it is ranked.

  • A story asks for something new.
  • A bug says something already agreed doesn’t work.
  • The right label keeps reports honest.
Read more: Story vs bug

A story asks for something new. A bug says something already agreed doesn’t work. Getting the label right decides how the work is ranked, and keeps the reports honest.

FeatureStoryBug
What it isNew or changed behavior the business wantsThe product doesn’t do what was already agreed
Who writes itThe product owner, with the teamAnyone who finds it: a tester, developer or user
Checked againstIts own acceptance criteria (the agreed rules for “done”)The acceptance criteria of the story it broke
Payments example“As a customer, I want to schedule a wire for a future date.”The scheduled wire is sent today instead of on the chosen date.
How it is rankedIn the backlog, by value against effortBy severity and priority

How epics, features and stories fit together: an epic is a big goal, such as “Domestic wires”. It splits into features, such as “Wire approvals” and “Scheduled wires”. Each feature splits into stories small enough to finish in one sprint. Link every bug to the story it broke, and you can see which feature and which epic are at risk.

The gray areas

  • “It was never in the requirements.” If the behavior was never agreed, it is a new story, not a bug, even if users expected it. Tag it as a requirement gap, so you can see how often it happens.
  • Found while the story is still in the sprint: the story isn’t done yet. Fix it as part of the story; many teams log it as a task on the story, not as a separate bug.
  • Found after the story is done (in a later sprint or in production): log a separate bug and link it to the story.
  • Keep the labels honest. Don’t call a bug a story to make the bug count look better, or a story a bug to skip the queue.

The numbers to track

The team tracks a few numbers to see whether bugs are under control.

  • Open bugs by severity and age.
  • Fix bugs found in this sprint in this sprint.
  • A live incident is fixed first by restoring service, then logged as a bug.
Read more: The numbers to track
NumberWhat it meansHow to work it out
Defect ageHow long a bug has been openToday (or the close date) minus the open date. Track it by severity.
Defect leakageBugs that slipped past a test levelBugs found after a test level ÷ all bugs found × 100. Example: 6 of 198 bugs were found in production = 3% leakage.
Bugs per sprintNew bugs found and fixed in each sprintCount both. Over time, fixed should beat found.
Bugs per storyHow many bugs each delivered story bringsBugs found ÷ stories delivered in the sprint. Falling over time is good.
Reopen rateFixes that failed the retestReopened bugs ÷ fixed bugs × 100
Production incidentsLive problems that hurt customersCount per month, with the time to restore service

Sprint bugs vs older bugs

Fix a bug found in the current sprint in the same sprint where you can. A story isn’t “done” while it has an open critical bug. Bugs from older sprints go to the backlog and are ranked with the rest of the work.

Production incident vs bug

A production incident is something broken in live right now, hurting customers. The first job is to restore the service (roll back, switch the feature off, or use a workaround), not to find the perfect fix. Then a bug is raised for the root cause and follows the normal life cycle. Not every incident is a code bug: it can be a partner outage, an expired certificate or bad data. When it is a bug, it counts as leakage, so ask which test level should have caught it.

Fix faster with AI, then retest

The developer uses an AI assistant to draft a fix for Maya’s bug. It’s faster, but a person still decides.

  • AI can draft the fix, the test and the notes.
  • Retest with the same steps and data.
  • The new test must fail on the old code.
Read more: Fix faster with AI, then retest

AI assistants, such as GitHub Copilot or an approved AI chat tool at your company, can speed up almost every step of the bug life cycle. They draft quickly, but a person still decides. Here is where they help, and the check that stays with a human:

StepHow AI helpsThe human check
Write the bugTurns logs, screenshots and notes into a clear title, steps, and expected vs actual resultThe tester confirms the steps really reproduce the bug
TriageFlags likely duplicates, and suggests a category, owner and severityThe triage lead decides severity and priority
Find the causeExplains error messages and stack traces, and points to the code that probably changedThe developer confirms the real root cause
FixSuggests a code change, plus a unit test for itA second developer reviews the code before it is merged
RetestTurns the bug’s steps into an automated test, and picks which regression tests to runThe tester checks the new test fails before the fix and passes after it
CloseDrafts the fix summary and the release noteThe bug owner reads it before closing

How to retest an AI-assisted fix

  1. Repeat the original steps with the same data, and confirm the bug is gone.
  2. Run the new automated test, and make sure it would have caught the bug: it must fail on the old code.
  3. Run the regression tests around the change, because AI fixes can quietly break something nearby.
  4. Test the payment unhappy paths the fix touches: a duplicate request, a return, a timeout. None may pay twice.
  5. Close with evidence: test results, screenshots or logs attached to the bug.

For the full checklist, with payment unhappy paths and a sample test, read How to retest an AI-assisted fix.

Guardrails for payments

  • Never paste real customer data, account numbers or production logs into a public AI tool. Use your company’s approved tool, with masked data.
  • Every AI-suggested fix gets the same code review as any other change. In payments, that review is often a required control.
  • Check AI-written tests carefully. A test can pass while checking the wrong thing.
  • Measure whether it helps: compare defect age and reopen rate before and after you start using AI.

Reports every test lead should share

Every week, the test lead shares a short report.

  • Start with the release status and the top risk.
  • One message per chart.
  • Pull the reports automatically from your bug tool.
Weekly test summary: week 8One page the whole team can read in a minute!Release status: Amber (at risk). 2 critical bugs to fix before go-live.Tests run412 of 45092% of planned testsTests passed38994% of tests runOpen bugs502 critical, 6 highDefect leakage3%6 of 198 bugs in productionReopen rate6%fixes that failed retestOldest open bug41 daysowner: core banking teamEpics on track2 of 3Domestic wires at riskFeatures ready5 of 72 blocked by critical bugsStories tested34 of 4085% of in-scope storiesNot ready: wire approvals and scheduled wires (Domestic wires epic)Top risk: wire approvals fail in UAT for amounts over $1M. Fix due Wednesday.Example data
Read more: Reports every test lead should share
ReportThe question it answersBest way to show it
Weekly test summaryAre we ready to release (go / no-go), and what is the biggest risk?One page: a status line, key numbers for tests and bugs, progress by epic, feature and story, and the top risk
Weekly trendAre we fixing bugs faster than new ones arrive?Line chart: opened vs closed
Defect leakage by phaseWhich test level let bugs through to production?Bar chart by test level, production highlighted
Defect ageWhich bugs have been open too long, and why?Bars by age group, oldest highlighted
Defects per sprintIs quality getting better sprint by sprint?Two bars per sprint: found and fixed
Defects by categoryWhere do our bugs come from? Use all 12 categories above.Bar chart, largest first
Defects by sourceWhich kind of testing finds our bugs: manual, automation, performance, security or compliance?Bar chart, largest first
Interface bugs by connectionWhich connection breaks most: upstream, downstream, external or legacy?Bar chart, largest first
Bugs by applicationWhich channel shows the most bugs: mobile, web, APIs or legacy?Bar chart, largest first
Defects by application and priorityWhich systems carry the most serious bugs?A table: applications down, priority across
Planned vs exploratory testingHow many bugs come from scripted tests compared with exploring?Two bars side by side

What the main reports look like

The charts below use example data. Keep each chart to one message, written as its title, and highlight only the bar that needs action. The samples for interface bugs by connection and bugs by application are in their own sections above.

Start with the weekly summary. Most readers only look at this page, so put the release status and the top risk first.

Weekly trend: opened vs closedHealthy when the closed line stays above the opened line0102030W1W2W3W4W5W6W7W8W1: 24 openedW2: 30 openedW3: 28 openedW4: 22 openedW5: 18 openedW6: 15 openedW7: 12 openedW8: 9 openedOpenedW1: 10 closedW2: 18 closedW3: 25 closedW4: 27 closedW5: 24 closedW6: 20 closedW7: 16 closedW8: 14 closedClosedClosing overtakes openingExample data
Where bugs were found (defect leakage)The earlier a bug is found, the cheaper it is to fix0204060Requirements: 18 bugs18ReqsUnit: 42 bugs42UnitSystem: 60 bugs60SystemSIT: 38 bugs38SITE2E: 20 bugs20E2EUAT: 14 bugs14UATProd: 6 bugs6ProdLeakage to production: 6 of 198 bugs = 3%Example data
Defect age: how long open bugs have waitedGroup by age, and show the oldest separately0–2 days0–2 days: 22 open bugs223–7 days3–7 days: 15 open bugs158–30 days8–30 days: 9 open bugs9Over 30 daysOver 30 days: 4 open bugs44 bugs open over 30 days: ask whyExample data
Bugs per sprint: found vs fixedFewer new bugs each sprint means quality is improving051015Sprint 1: 14 bugs foundSprint 1: 9 bugs fixedS1Sprint 2: 12 bugs foundSprint 2: 11 bugs fixedS2Sprint 3: 15 bugs foundSprint 3: 14 bugs fixedS3Sprint 4: 9 bugs foundSprint 4: 12 bugs fixedS4Sprint 5: 7 bugs foundSprint 5: 9 bugs fixedS5Sprint 6: 6 bugs foundSprint 6: 8 bugs fixedS6Bugs foundBugs fixedFrom sprint 4, fixes outpace new bugsExample data
Defects by categoryAll 12 categories, largest first; the top bar is where to actFunctionalFunctional: 58 bugs58DataData: 30 bugs30EnvironmentEnvironment: 24 bugs24RequirementRequirement: 22 bugs22InterfaceInterface: 15 bugs15Usability and contentUsability and content: 14 bugs14CompatibilityCompatibility: 10 bugs10PerformancePerformance: 9 bugs9SecuritySecurity: 6 bugs6AccessibilityAccessibility: 5 bugs5Not reproducibleNot reproducible: 3 bugs3Tester errorTester error: 2 bugs2Functional: 58 of 198 bugs (29%)Example data
Defects by sourceShows which kind of testing finds your bugsManual testingManual testing: 90 bugs90AutomationAutomation: 68 bugs68PerformancePerformance: 14 bugs14ComplianceCompliance: 11 bugs11SecuritySecurity: 9 bugs9ProductionProduction: 6 bugs6Automation found 68 of 198 bugs (34%)Example data

Defects by application and priority works best as a simple table (all 198 bugs, example data):

ApplicationCritical and highMedium and low
Mobile app955
APIs and backend849
Web546
Legacy323
Total25173

Read it across: the mobile app has the most bugs, but the APIs carry almost as many serious ones.

Planned vs exploratory testingBugs found by scripted tests compared with exploringPlanned (scripted)Planned (scripted): 140 bugs140ExploratoryExploratory: 58 bugs58Exploratory testing found 58 of 198 bugs (29%)Example data

Always map defects to a root cause (the real reason behind it). That is what turns a bug list into fewer bugs next time.

Get these reports automatically

Most test leads still build these reports by hand in Excel every week, often pulling data from two or three tools. I’m building a small app that does it for you. It reads your bugs and test runs from Jira, qTest, ALM or a CSV export, and produces every report on this page, plus the go / no-go view below.

  • Map once: each tool names things differently (“Blocker” in one, “1-Critical” in another). You map your values once, then every report works the same.
  • Read only: it never changes anything in your tools.
  • Safe for payments: card and account numbers in bug titles are masked before anything is stored.

KPIs and the go / no-go decision

Finally, the team decides: can the release go live?

  • Agree the exit criteria with the business up front.
  • Three choices: go, go with conditions, or no-go.
  • Decide with the data, not the mood in the room.
Read more: KPIs and the go / no-go decision

A KPI (key performance indicator) is a number with a target that shows whether you are on track. Before a release, a few KPIs become the exit criteria: the conditions that must be true to go live. Agree them with the business, operations and compliance before testing starts, so the decision is about facts, not feelings.

KPITarget to go live (example)Week 8 result
Epics ready for releaseAll in-scope epics2 of 3: not met (domestic wires)
Features ready for releaseAll in-scope features5 of 7: not met (wire approvals and scheduled wires)
Stories passed acceptance testsAll in-scope stories34 of 40: not met
Open critical bugs02: not met
Open high bugsOnly with an agreed workaround6, all with workarounds: met
Planned tests runAt least 95%92%: not met
Pass rateAt least 95% of tests run94%: not met
Compliance and security bugsAll closedAll closed: met
UAT sign-offSigned by the businessWaiting on the 2 critical fixes
Rollback and penny test plansWritten and rehearsedDone: met

The three possible decisions

  • Go: every exit criterion is met.
  • Go with conditions: a few criteria are missed, but the risk is understood and small. A feature that isn’t ready can sometimes be switched off and released later, while the rest goes live. The known issues, their workarounds and a fix date are written down, and the business owner accepts them in writing.
  • No-go: a critical criterion is missed, such as an open critical bug or missing sign-off. Set a new date and say exactly what must be true by then.

Week 8 in this example is a no-go: two critical bugs are open. The test lead recommends a new date three days later, once both fixes are retested and UAT is signed.

Who decides

The test lead brings the evidence and a clear recommendation. The release owner (often the product owner or release manager) makes the call, with the business, operations and compliance in the room. In payments, any one of them should be able to stop a release that puts customers’ money or a regulatory rule at risk.

My take

A bug that is clearly explained, honestly prioritized and seen by the right people gets fixed. Treat every defect report as a short pitch, and your fix rate will show it.

Working on a payments project?

I help banks and fintechs deliver real-time payments, ISO 20022, and AI-driven AML and compliance change, from strategy to go-live.

Connect on LinkedInSee how I can help