Still Required, Not Admired: Traditional Pen Tests

In my role I’m fortunate to talk to and learn from a number of experienced CISOs. Unequivocally, they tell me that traditional penetration (pen) testing isn’t rapid enough or comprehensive enough to evaluate an organization’s entire attack surface. “Pen tests are stale bread,” is how one likes to put it. Another theme for these CISOs is that mandated regulatory requirements for pen testing aren’t keeping pace with today’s accelerated attacker risk. Read on to find out why the human-led pen test is a security tool that should be an “and” at best, not an “instead of” more comprehensive testing. 

The Reasons for Pen Testing

There are two key reasons organizations conduct traditional human-led penetration tests:

• To identify weaknesses that will help them improve their security posture
• To fulfill regulatory mandates

Recent research we did with Dark Reading shows that current enterprise pen testing practices are driven more frequently by a desire to improve cybersecurity than to fulfill compliance requirements. In fact, the top two reasons that security professionals told us they conduct penetration tests are to measure their security posture and prevent breaches, with meeting regulatory requirements coming in third.

That’s somewhat surprising to me for two reasons. First, many security and compliance frameworks, like the NIST 800-53: Security and Privacy Controls for Information Systems and Organizations and the Financial Industry Regulatory Authority (FINRA), dictate the use of periodic penetration testing in conjunction with vulnerability scanning to achieve compliance. Second, it’s also surprising given the predictions of pen testing’s demise over the last 15 years and the devaluation of the pen test by many CISOs, even those who started their careers as pen testers. 

Do Pen Tests Make You More Secure?

But the fact remains that most enterprises spend hundreds of thousands of dollars on penetration tests annually. Some spend millions! Let’s explore how and whether different approaches to pen testing can achieve the intended purpose of making organizations significantly more secure. 

The traditional pen test is typically approached as a deep dive into a scoped segment of the IT ecosystem. A vulnerability scan of the defined scope is often the first step in the process; a final report of a potential attack path developed over a period of weeks is the typical deliverable. 

Pen tests are deep but narrow, time-consuming, expensive and highly variable in the insights they deliver. The variability may be due to the scope of the assignment, the budget allocation, and certainly the training and quality of the individual pen tester. It’s often said that a pen test is an inch wide and a mile deep, or as deep as the pen tester’s skills. 

A skilled pen tester, aka ethical hacker, will deploy techniques that attackers can use and machines can’t. These include social engineering practices to obtain credentials; loitering outside buildings with smokers to gain physical access and other ingenuous ploys. At its best, a traditional pen test draws on human insight and maneuvers to illuminate how vulnerabilities can be chained together. But many pen tests don’t reach that level of ingenuity. In private, CISOs divulge that some lower-level pen testers may deliver little more than Metasploit output.