
⏱ 2-minute read · open “Read more” for the details
In Part 1, I explained what PCI DSS (security rules for anyone who handles card data) is, the 12 requirements, the merchant levels, and how it fits with the card brands, Apple Pay and PayPal.
PCI DSS is the security standard for anyone who handles card numbers. Part 2 explains what actually happens in a PCI audit, how to measure whether your checks really work, and what software teams must do.
This is Part 2: PCI DSS in practice. What really happens in an audit, how to measure whether your security tools find real problems, what software teams must do, and why it all matters to the customers paying you.
The audit view: what really happens
Meet Leo, a product manager at an online store that takes cards. The PCI audit is coming. Here is what really happens.
- Larger businesses are assessed by a Qualified Security Assessor (QSA) or a certified internal assessor (ISA).
- The assessor checks evidence, interviews people and samples systems.
- Since v4.0, you can follow the defined approach or a customized one.
Read more: The audit view: what really happens
For larger businesses, proof means a formal assessment by a Qualified Security Assessor (QSA) or a certified internal assessor (ISA). Here is what it looks like from the inside.
| When | What happens |
|---|---|
| Before | Agree the scope with the assessor (which systems and people the audit covers). Prepare evidence: network and data-flow diagrams, policies, system settings, logs, training records, and your vendors’ own AOCs (signed proofs that a company passed PCI). |
| During | The assessor interviews your people, watches processes in action, reads documents and checks system settings. They test samples, and under v4.0.1 they decide how big the sample is and must explain why in the report. |
| Findings | Each requirement gets a result. Gaps found and fixed during the assessment are recorded as “in place with remediation” (fixed during the audit), so the fix is visible, not hidden. |
| After | The assessor signs the Report on Compliance (ROC). You sign the Attestation of Compliance (AOC) and send it to your bank, and to the card brands if they ask. |
Two ways to meet a requirement. Since v4.0, you can follow the defined approach, doing exactly what the requirement says, or the customized approach, meeting the requirement’s goal in your own way. The customized approach needs a written risk analysis for that requirement and solid evidence, and it’s only allowed with a full ROC, not a self-assessment. The assessor who helped you design a customized control can’t be the one who tests it. And not every requirement can be customized.
Why audits go badly:
- Scope surprises: card numbers turn up in logs, emails, spreadsheets or call recordings nobody knew about.
- Missing evidence: the control exists, but nobody can prove it ran every quarter.
- Policy on paper only: the document says one thing, and staff do another.
- Vendors without proof: a third party touches card data but can’t show a current AOC.
Auditor tip: treat evidence like a product. Collect it automatically, all year, in one place. The best audits are boring.
Measuring findings: true, false and missed
Leo’s tools raise lots of alerts. Which ones are real?
- True positive: a real problem, caught.
- False positive: an alarm with no real problem.
- False negative: a real problem, missed. This is the most dangerous.

Read more: Measuring findings: true, false and missed
Every PCI control produces findings: vulnerability scans, card data discovery tools, log alerts, alerts when someone changes your payment page, penetration tests and the audit itself. But not every finding is real, and not every real problem gets found. There are three kinds:
Each box answers two questions. Is there really a problem? Did the tool say so? Dark blue (top left) is what you want: a real problem caught. The box with the thick border (top right) is the dangerous one: a real problem the tool missed. Bottom left is a false alarm that wastes time. Bottom right means nothing is wrong and nothing was flagged.
| Type | What it means | Example | The risk |
|---|---|---|---|
| True finding (true positive) | A real problem, correctly flagged | The scan finds a server missing a security patch | Must be fixed and retested |
| False finding (false positive) | Flagged, but not a real problem | The scanner flags a fixed software version; a discovery tool mistakes an order number for a card number | Wastes time and causes “alert fatigue”, so real alerts get ignored |
| Missed finding (false negative) | A real problem nobody flagged | Card numbers sitting in call recordings that no tool scanned | The most dangerous: you think you’re safe, but you aren’t |
The metrics to track every quarter:
| Metric | How to measure it | What good looks like |
|---|---|---|
| True finding rate | True findings ÷ all findings flagged | Rising, as tools are tuned |
| False finding rate | False findings ÷ all findings flagged | Falling over time |
| Missed findings | Real issues found by pen tests, audits or incidents that your tools didn’t catch | Zero. Each one becomes a new rule or check. |
| Time to triage | Time from alert to “true or false” decision | Days, not weeks |
| Time to fix | Time from confirmed finding to fixed and retested, by severity | Critical issues fixed fastest |
| Repeat findings | The same issue coming back quarter after quarter | Zero |
How false findings are handled in PCI. You can’t simply dismiss your own findings. For the quarterly external scans, you raise a dispute with your Approved Scanning Vendor (ASV: an approved company that scans your systems for weak spots) and back it up with evidence, such as configuration screenshots or vendor advisories. The ASV decides. A scan only passes when no flaw scored 4.0 or higher remains. The score comes from CVSS, a 0 to 10 scale for how serious a security flaw is. The evidence must be checked again for each scan period, not carried forward automatically.
An example: a card data discovery scan flags 1,000 “possible card numbers”. Triage shows 50 are real and 950 are order numbers and test data. That’s a 5% true finding rate and a 95% false finding rate. The 50 real ones are a scope surprise to fix now. The 950 false ones show the tool needs tuning, for example by checking card number formats and ignoring known order-number fields.
Tip: don’t tune a tool so hard that it goes quiet. A tool with zero false alarms may also be missing real problems. If you work in AML or fraud, these are the same measures used to judge alert quality: precision, false positives and missed cases.
The software view: building payment software that passes
Leo’s team also builds payment software, which has its own rules.
- PA-DSS was retired in October 2022.
- It was replaced by the PCI Secure Software Standard and the Secure Software Lifecycle Standard.
Read more: The software view: building payment software that passes
PCI DSS covers the company. A separate set of rules covers payment software. The old software standard, PA-DSS, was retired on 28 October 2022 and replaced by the PCI Software Security Framework, which has two parts:
- PCI Secure Software Standard: security requirements for a payment software product itself.
- PCI Secure Software Lifecycle (Secure SLC) Standard: requirements for how a software vendor builds and maintains its software, from design to end of life.
The Council lists validated payment software and qualified vendors on its website, so when you buy payment software, check the list and ask for proof.
For your own development teams, these habits keep PCI built in, not bolted on:
- Train developers in secure coding every year, and review code for security before release.
- Never write full card numbers to logs, error messages, analytics or support tickets.
- Use test cards and tokens in test environments, never real card data.
- Keep an approved list of every script on your checkout pages, and alert on any change.
- Scan for vulnerabilities, patch quickly, and keep a list of all third-party libraries.
- Store card data only where truly needed, encrypted, with keys managed separately.
A checklist for payments product teams
Here is Leo’s short checklist.
- Add PCI checks to the definition of done.
- Review scope when you add a vendor, system or payment flow.
- Collect evidence all year, not just before the audit.
Read more: A checklist for payments product teams
- Add PCI checks to your definition of done for every story that touches card data.
- Review your scope whenever you add a new vendor, system or payment channel.
The customer view: why it matters to the people paying
Customers never see PCI DSS, but they feel it every time they pay.
- In the US, federal law limits a cardholder’s loss from unauthorized charges.
- Visa and Mastercard go further with zero-liability policies.
- That trust is what brings customers back.
Read more: The customer view: why it matters to the people paying
Customers never see PCI DSS, but they feel it every time they pay. Here’s the surprising part: in the US, the customer usually doesn’t lose money from card fraud.
- Federal law limits a cardholder’s liability for unauthorized credit card use to $50.
- Visa and Mastercard go further with zero liability policies, so customers generally pay nothing for fraudulent charges they report.
So who pays when card data is stolen? The businesses and banks in the chain: refunds and chargebacks (when banks take money back from sellers for their customers), replacing cards, investigations, fines and lost customers. That’s why PCI DSS exists. Customers are protected from the loss, so the companies must protect the data.
What customers do notice:
- Trust at checkout: familiar, secure payment options like Apple Pay, Google Pay or PayPal.
- Masked card numbers: only the last four digits shown on screens, receipts and emails.
- No surprises: nobody asks for a card number over email or chat.
- Fast, honest communication if something ever goes wrong.
The customer test: if your customers knew exactly how you handle their card details, would they feel safe? If the answer is yes, you are on the right track.
My take
Passing PCI DSS isn’t about a busy month before the assessor arrives. It’s about evidence collected all year, findings measured honestly, software built safely, and customers who never have to think about it. Do that, and the assessment becomes a formality instead of a fire drill.
Working on a payments project?
I help banks and fintechs deliver real-time payments, ISO 20022, and AI-driven AML and compliance change, from strategy to go-live.
Connect on LinkedInSee how I can help