Still Required, Not Admired: Traditional Pen Tests

By Dixie Fisher, Senior Product Marketing Manager | April 7, 2021

In my role I’m fortunate to talk to and learn from a number of experienced CISOs. Unequivocally, they tell me that traditional penetration (pen) testing isn’t rapid enough or comprehensive enough to evaluate an organization’s entire attack surface. “Pen tests are stale bread,” is how one likes to put it. Another theme for these CISOs is that mandated regulatory requirements for pen testing aren’t keeping pace with today’s accelerated attacker risk. Read on to find out why the human-led pen test is a security tool that should be an “and” at best, not an “instead of” more comprehensive testing. 

The Reasons for Pen Testing

There are two key reasons organizations conduct traditional human-led penetration tests:

  • To identify weaknesses that will help them improve their security posture
  • To fulfill regulatory mandates

Recent research we did with Dark Reading shows that current enterprise pen testing practices are driven more frequently by a desire to improve cybersecurity than to fulfill compliance requirements. In fact, the top two reasons that security professionals told us they conduct penetration tests are to measure their security posture and prevent breaches, with meeting regulatory requirements coming in third.

That’s somewhat surprising to me for two reasons. First, many security and compliance frameworks, like the NIST 800-53: Security and Privacy Controls for Information Systems and Organizations and the Financial Industry Regulatory Authority (FINRA), dictate the use of periodic penetration testing in conjunction with vulnerability scanning to achieve compliance. Second, it’s also surprising given the predictions of pen testing’s demise over the last 15 years and the devaluation of the pen test by many CISOs, even those who started their careers as pen testers. 

Do Pen Tests Make You More Secure?

But the fact remains that most enterprises spend hundreds of thousands of dollars on penetration tests annually. Some spend millions! Let’s explore how and whether different approaches to pen testing can achieve the intended purpose of making organizations significantly more secure. 

The traditional pen test is typically approached as a deep dive into a scoped segment of the IT ecosystem. A vulnerability scan of the defined scope is often the first step in the process; a final report of a potential attack path developed over a period of weeks is the typical deliverable. 

Pen tests are deep but narrow, time-consuming, expensive and highly variable in the insights they deliver. The variability may be due to the scope of the assignment, the budget allocation, and certainly the training and quality of the individual pen tester. It’s often said that a pen test is an inch wide and a mile deep, or as deep as the pen tester’s skills. 

A skilled pen tester, aka ethical hacker, will deploy techniques that attackers can use and machines can’t. These include social engineering practices to obtain credentials; loitering outside buildings with smokers to gain physical access and other ingenuous ploys. At its best, a traditional pen test draws on human insight and maneuvers to illuminate how vulnerabilities can be chained together. But many pen tests don’t reach that level of ingenuity. In private, CISOs divulge that some lower-level pen testers may deliver little more than Metasploit output.

Regardless of quality, it’s critical to note that pen testing is best considered an “and” to your approach to security testing; never an “instead of” more comprehensive testing.

Some History

At its inception in the 1960s, the goal of penetration testing was to show how ostensibly well-designed architectures could be artfully breached. The findings of this probing, human-led exercise would be shared back with the security architects to help them improve their security by design. Just over 50 years ago, it was about “if” a malicious actor could penetrate your defenses. Now we all know it’s not if, but when… and where. 

In the 21st century, you don’t need a pen test to show you your network can be breached. You only need to read the headlines to learn that highly reputable companies with well-funded security budgets — even those who sell cybersecurity products! — can and have been breached. 


Where the Pen Test Fits Today

Traditional pen tests must be an “and” to a good cybersecurity approach because they can never be executed broadly, frequently and fast enough to reveal attackers’ many paths into a network regardless of your budget allocations.

Even if you were somehow able to create a virtuous cycle of constant pen tests, they would still fail in revealing all possible attacker paths, because they — even in combination with vulnerability scanning — aren’t being executed by starting with a full discovery of your IT ecosystem as the foundation. By their very nature pen tests are scoped and executed with guard rails, so they can’t, don’t, won’t find all the risks in your extended IT ecosystem; that’s just not what they are designed to do. 


CISO’s Opinion: “Pen Tests Are Stale Bread”

For this reason, many CISOs are outspoken in opposition of pen tests as a required security technique and are waiting for regulatory bodies to catch up with modern security realities. Candid CISO comments shared with us echo that:

  • “Pen tests are stale bread, by the time you get the reports, they are out of date.”
  • “Penetration testing has not kept pace with increasingly sophisticated attacks, it’s really time to retire pen testing.”
  • “Manual processes work at the speed of humans, but the Internet is moving at the speed of machines.” 

The Path Forward

A better way to test the foundation of your security program is with comprehensive and continuous testing of your extended attack surface, say these CISOs. With that foundation, you can comfortably layer on other approaches, including pen tests, in areas of greatest risk. Some CISOs use the attack surface discovery and risk prioritization of the CyCognito platform to determine the scope and methodology of their mandatory pen tests. Other CISOs are replacing discretionary penetration testing with baseline continuous testing by the CyCognito platform and their auditors accept that. CISOs overall are looking for better, more comprehensive ways to secure their company’s entire attack surface and gain better risk reduction from their security dollars. The evolution of testing methods for continuous proactive security isn’t an “and” or an “instead of,” it’s a must have. 

To learn more about the penetration testing and other alternatives, see the White Paper: Rethinking Penetration Testing.

About Dixie Fisher, Senior Product Marketing Manager

Dixie Fisher, Senior Product Marketing Manager, has been marketing innovative security solutions and best practices for over a decade.

Contact Author:
  • linkedin
  • email

Start Eliminating Your Shadow Risk

Demo Request