Security Validation: Continuous Testing of Defenses

Security validation is the continuous, evidence-based practice of proving whether your controls actually stop the attacks that will come at you. It isn't a tool or an annual report: it's a programme that runs real attack techniques against your environment in a repeatable way, measures what gets prevented, what gets detected and what goes completely unnoticed, and turns every gap into a concrete improvement. The question it answers isn't "do I have controls deployed?" but "do those controls stop the adversary when they try?". That distinction matters: most organisations discover that their EDR, their SIEM and their firewall rules perform far worse in practice than the admin dashboard suggests.

Key takeaways on security validation

• It validates with evidence whether your controls stop real attacks, not whether they are merely installed.
• It's a continuous programme, not a one-off exercise: it measures prevention, detection and response over time.
• Gartner groups it under Adversarial Exposure Validation (AEV), the validation stage within CTEM.
• BAS (Breach and Attack Simulation) is a method inside the programme, not a replacement for pentesting or Red Team.
• Everything is anchored to MITRE ATT&CK so you speak a common language of tactics and techniques.

What security validation is

Security validation is the discipline that empirically demonstrates the effectiveness of your defences by executing attacker behaviour and observing how your security stack reacts. Instead of assuming a control works because the vendor promises it or because it passes a configuration review, you put it to the test: you launch the technique, you record whether the control blocks it, whether it raises an alert, and whether that alert reaches an analyst who acts.

The industry has coined several terms around this idea. Gartner consolidated the market under the name Adversarial Exposure Validation (AEV), which merges BAS and automated pentesting, and positioned it as the validation stage within CTEM (Continuous Threat Exposure Management), the framework it proposed in 2022 to manage exposure continuously. In parallel, NIST CSF 2.0 insists that the Protect, Detect and Respond functions are worthless unless they are tested on a recurring basis.

The conceptual key is this: validation isn't hunting for new vulnerabilities like a scanner, it's confirming whether your existing controls do their job against the tactics, techniques and procedures (TTPs) a real adversary would use against your sector.

Validation as a programme, BAS as a method, pentest as a point in time

This is where most of the commercial confusion happens. Security validation is the umbrella; BAS, pentesting and Red Team are methods that the umbrella orchestrates at different cadences and depths. They do not compete: each plays a distinct role and the programme decides which one applies at any given moment.

BAS (Breach and Attack Simulation) is the piece that supplies the automated breadth. In that guide we break down its technology, its platforms and the fine-grained comparison against the pentest and the Red Team. Here we go up a level: what sets a validation programme apart is not the tool that runs the attacks but the governance and the metrics that orchestrate those pieces into a repeatable cycle, deciding when automated breadth applies and when manual depth does. If you want the detail of when to choose each offensive method, we develop it in pentesting vs Red Team.

The four pillars of a validation programme

Continuous security validation

Continuous security validation applies the principle that environments change daily: a new rule is deployed, a certificate expires, an EDR agent stops reporting, someone opens a port. This control drift silently degrades your defence. Continuous validation runs checks at a high frequency (daily or weekly) to catch that degradation before an attacker exploits it.

Automated security validation

Automated security validation is the engine that makes continuity viable. Without automation, repeating hundreds of techniques every week is impossible in human hours. This is where BAS and automated pentesting platforms live, launching safe-by-design payloads and collecting the result with no manual intervention.

Security control validation

Security control validation focuses on a specific control: does my EDR detect reflective DLL loading? Does my proxy block DNS exfiltration? Does my WAF stop this injection? Each MITRE ATT&CK technique is mapped against the control responsible for it, producing a matrix of real coverage rather than theoretical coverage.

Security posture validation

Security posture validation aggregates all of the above into a whole-of-organisation picture: what percentage of the techniques relevant to your sector are prevented, which are only detected, and which pass without a trace. It's the metric a CISO takes to the board to justify investment.

How it fits with CTEM and adversarial exposure

CTEM defines five stages: scoping, discovery, prioritisation, validation and mobilisation. Security validation is the fourth and, for many, the most neglected. A perfect inventory of vulnerabilities is of little use if you don't know which are actually exploitable in your context or whether your controls would contain them. Adversarial validation cuts the noise: instead of remediating 3,000 CVEs by their theoretical CVSS score, you prioritise the few that an attacker could chain into real impact, which you only learn by testing it.

The programme does not live in isolation inside that fourth stage: it feeds the others. Validation results reorder the prioritisation (a vulnerability your EDR already blocks drops in urgency; one that slips through unnoticed jumps to the top) and they trigger mobilisation whenever a confirmed gap demands a new rule or a configuration change. That continuous loop, not a quarterly snapshot, is what turns CTEM into genuine exposure management. When the validation stage leans on BAS for the automated part, the cycle can run in days rather than months.

How to build the programme step by step

1. Start from the threat, not the tool. Define which groups and TTPs are relevant to your sector using threat intelligence and the MITRE ATT&CK framework. Validating techniques nobody will use against you is wasted budget.
2. Choose the methods by objective. Automated BAS for breadth and continuity, Purple Team for collaborative depth between attacker and defender, Red Team for the final unannounced exam.
3. Assemble the toolkit, but do not fixate on it. The specific tool choice (open-source engines like Atomic Red Team or Caldera and commercial platforms like Cymulate, AttackIQ or Picus) is compared in depth in the BAS guide. At programme level the brand matters less than ensuring every executed technique has its defensive counterpart: detection is versioned as code with Sigma rules so the offensive catalogue and the detection catalogue evolve in step.
4. Run on a cadence. Continuous checks are automated daily or weekly; purple team monthly or quarterly; red team annually. Regularity is what turns loose tests into a programme.
5. Close the loop. Every technique that went undetected must generate a new rule in your SIEM or SOAR and a verification that it now fires. Without that closure, validation only produces lists of bad news.

The work of turning findings into new detections falls to the Blue Team and to detection engineering. Validation without that receiving team is an exam nobody grades.

Metrics and evidence

A serious programme is measured, not narrated. The reference metrics are MTTD (mean time to detect), MTTR (mean time to respond), the percentage of ATT&CK technique coverage prevented or detected, and the prevention versus detection rate (blocking beats merely seeing). Just as important is watching for drift: 80 percent coverage that drops to 60 in three months signals controls that have degraded without anyone noticing.

Evidence is what separates validation from an opinion: alert identifiers, timelines, screenshots and session logs that prove, technique by technique, what happened. To validate the effectiveness of a specific EDR, the MITRE Engenuity ATT&CK Evaluations offer a public, independent reference for how different products detect groups such as APT29.

Common mistakes that erode value

• Confusing validation with an annual pentest. One exam a year validates nothing the rest of the year, which is exactly when controls change.
• Validating prevention only. If you assume breach, what matters is what you detect when prevention fails, not just how much you block.
• BAS without intelligence. Firing thousands of techniques without prioritising by real threat produces noise and fatigue.
• Not closing the loop. A finding with no new detection and no verification is wasted work.

Frequently asked questions

Does security validation replace pentesting or Red Team?

No. It integrates them. Pentesting brings manual depth on specific assets and Red Team brings the honest test of detection under real pressure. Security validation is the programme that decides when to apply each, adds automated BAS for continuous breadth, and measures the whole over time. Replacing the point-in-time exam with automation would be as wrong as keeping only the annual exam.

What is the difference between BAS and security validation?

BAS (Breach and Attack Simulation) is a technology: it automates the repeatable execution of known attack techniques against your environment. Security validation is the broader discipline that uses BAS as one of its methods, alongside purple team and red team, and that also defines the strategy, cadence, metrics and loop closure. BAS is the engine; validation is the whole vehicle.

How often does a validation programme run?

It depends on the layer. Automated checks of critical controls can run daily or on every relevant change. Purple team exercises tend to be monthly or quarterly. A full red team is reserved for once or twice a year. What defines a programme rather than an isolated test is precisely that sustained cadence.

Who should own the validation programme?

Executive accountability usually sits with the CISO, but the programme only works with two coordinated hands: an offensive team (internal or external) that runs the techniques and a Blue Team that turns every gap into detection. Governance defines who prioritises the threats, who approves the execution windows and who verifies that the loop closes. Without a clear owner answerable to the board, validation degenerates into a stream of reports nobody acts on. The choice between open-source and commercial BAS tooling, a more tactical decision, is compared in the BAS guide.

Does security validation help with NIS2 or DORA?

Yes. Both frameworks require proving the effectiveness of measures, not just declaring them. Continuous validation generates the auditable evidence that controls work, and in the case of DORA it connects directly to intelligence-led TLPT red team exercises for designated financial entities.

Next step

A security validation programme only delivers value if someone designs it with a threat-informed criterion and closes the loop all the way to detection. At Secra we build and operate that cycle: adversary emulation aligned with MITRE ATT&CK, purple team exercises with your Blue Team, and recurring validation that turns every gap into a verified detection rule. If you want to know where your detection stands today, tell us the context and we'll tell you where to start.