Common security testing approaches leave gaps. Here’s how to find them.

Gaps in your security testing program are likely more than simply missed assets. Infrequent testing and even low test accuracy are also gaps, and can be just as bad or worse.

Gaps happen despite the best efforts of everyone involved. The good news is that, with some strategic adjustments, you can reduce gaps using tools you likely already have deployed. 

Visualize Your Security Testing Timeline

In order to make improvements you need to first understand where you are. An effective way to assess your security testing is by thinking of it as woven cloth. IP addresses and web apps are threads running horizontally. Test schedules are threads running vertically. The intersections represent critical points in your network where testing occurs for each asset.

Figure 1 is an example, with circles, squares and triangles to represent network vulnerability scanning, dynamic application security testing (DAST) and penetration testing.

Figure 1 – Ten Week Security Testing for Seven Assets

In this example, App 2 and IP address 192.168.1.3 aren’t tested at all. 192.168.1.2 is missing vulnerability scanning at week 6 and week 8. And 192.168.1.4 is tested on a much different frequency than the rest. This is the what. In order to understand the why, you will need to dig further into the business function of these assets and review test policies.

A well-coordinated testing program resembles a tightly woven blanket, where most intersections are covered by colored dots. What you don’t want is a testing program that resembles a worn-out rug; great when new, but worn thin and no longer fit for its purpose.

Coverage, Frequency and Accuracy Gaps = Holes in the Cloth

Gap management is a constant juggling act. Your security teams have to adjust asset coverage, test frequency and test accuracy; it’s not uncommon to address only one or two out of the three (or none).

It’s not just your company. In a 2023 survey of 304 InfoSec teams, 81% rate security testing as important but fail to meet expectations for risk management. Gaps aren’t just the result of budget or time issues. Tool complexity, lack of integration, staff skills differences, and fragmented communication are the realities of today’s large IT security teams; any of which can contribute to unwanted risk. 

Benefits of Reducing Security Testing Gaps

By removing gaps, you can build greater efficiency into your processes, leading to several key advantages:

• Improved risk management. Frequent testing and full coverage are essential for addressing potential vulnerabilities promptly and lowering your mean time to remediation (MTTR).
• Better decision making. Full testing provides data that informs strategic decisions and builds a confident answer to “how protected are we” board-level questions.
• Accurate KPIs. Executive leadership needs accurate exposure measurements – this is only possible with if security testing gaps are low.
• Cost efficiency. Early detection and remediation of security issues are considered less expensive than dealing with breaches or vulnerabilities after they have been exploited.

You’ve Most Likely Taken These Approaches

Let’s use the three previously mentioned testing approaches as the basis of an example. According to Fortra’s 2023 Penetration Testing Report, network vulnerability scanning is deployed in 87% of surveyed organizations and penetration testing is deployed in 69%. CyCognito’s 2024 State of Web Application Security Testing Report found DAST is utilized on roughly 50% of deployed web apps.

InfoSec teams attempting to reduce gaps with these approaches likely do the following:

• Adjust the coverage and frequency of vulnerability scans; tune policies periodically.
• Carefully expand DAST for more web apps, some running in production.
• Broaden the scope of red teams/penetration testing or add (occasional) bug bounties. 

The goals are spot on. The challenge is in execution. A sound gap reduction strategy involves layered, coordinated testing. Test coverage, frequency, and accuracy need to be considered simultaneously, not independently. Without coordination, your efforts create more test data but you won’t know where the gaps are nor inform your next path of action should your budget or priorities change.

Instead, Use This Five Step Workflow

A methodical approach to reducing gaps pays dividends long term. Repeat this workflow monthly or quarterly to ensure gaps remain minimized.

1. Gauge your CMDB confidence. Assess your confidence in your asset database, focusing on exposed IP addresses and web apps. 

Why? Asset management is difficult. Only 30% of IT ops and security teams express 80-95% confidence in their network connected endpoints (Tanium, 2023); others say CMDB’s are often only 60% accurate. 

If you have low confidence in your CMDB, consider it a red flag for your entire gap reduction effort (and let’s be honest, low confidence means less than 95%).

2. Gather testing data. This includes the scope and cadence of tests as well as the IPs, subnets and web apps in which they are applied.
3. Map your security testing on a timeline. Plot assets (by IP or app FQDN), test frequency (by week), and test type. A spreadsheet or python pyplot works well for this.
4. Find the gaps. Carefully evaluate all intersections. Look not only for assets without testing but also long gaps between tests and mismatched test types. You may need to break up or group the data to make it more consumable.
5. Adjust your testing. Focus on your most serious gaps first. Maybe a new vuln scan policy in the third week of the month reduces your gap by 30%. Or a DAST policy is targeting the wrong system. Or your pen testing and bug bounties miss a set of critical assets completely. 

Want to move to the next level? Add service and business context to the asset in order to inform your priorities. For example, IP address 192.168.1.3 may deliver a web application (port 80/443) and FTP (port 21/990). List these services along with an estimate of business impact – for example, the web app may handle e-commerce or support a business-critical API.

Reduce Gaps Through Smart, Coordinated Security Testing

With a well-coordinated testing approach, you can turn your security testing fabric into a tightly woven defense, reducing your risk and improving your organization’s resilience against cyber threats.

What are your testing gaps? Answer a few questions about your use of three security technologies and receive a customized report using the CyCognito security gap calculator.

---

CyCognito helps Fortune 500 organizations eliminate security testing gaps in their external attack surface through continuous discovery, monitoring and active security testing, delivered as a fully automated service.