Demystifying Security Validation Technologies: What You Need to Know About Pen Testing

You can’t predict the future, but you can prepare for it. When it comes to IT (and OT), security validation is the closest to predicting the future as you can get. This is the second post in our series “Demystifying Security Validation Technologies: What You Need to Know,” in which we break down a number of security validation methods available today, provide the strengths and weaknesses of each, and explain how each functions in different IT environments. We’ve covered manual penetration testing; now it’s time to take a look at its close cousin, automated penetration testing.

What is Automated Penetration Testing?

As the name suggests, automated pen testing involves a higher degree of automation and speed than traditional pen testing. However, despite any claims of “fully automated penetration testing” that you may see in the market, security practitioners must generally guide the process, targeting, and calibration of automated pen tests.

Still, automated pen testing is useful in augmenting human cybersecurity capabilities. Automated pen tests are able to scale monitoring and control validation in comparison to traditional pen tests. This enables a more frequent cadence of testing and provides more rigorous quality control and standardization. 

Automated pen testing products include a suite of automated tools to scan for various known vulnerabilities across a wide array of software, devices, and endpoints. Some newer automated pen testing solutions even include some ability to test for vulnerabilities of APIs, though that’s still an emerging capability. 

Automated Pen Testing Strengths and Weaknesses

Many of the strengths of automated pen tests mirror those of traditional ones, though the primary advantage is that they can cover more of the IT environment and can be run more frequently and continuously. This enables better longitudinal findings and security control performance metrics. They can even save on resources, since internal teams can be trained to perform the testing exercises. 

But if resources are a concern (and in this industry,  we know that they always are) both of these advantages are a double edged sword. Because pen tests can never be fully automated, human analysts must validate each finding and research recommended remediation, aside from the most basic fixes. This has the potential to create more work for security teams if the signal-to-noise ratio is poor and indicated vulnerabilities do not pose a risk to the organization. 

Another unfortunate flaw of automated pen testing is that these tests would disrupt operations if the tools are run in live IT environments. To avoid this, most automated pen tests are run against non-production environments. As a result, those tests may lack fidelity, since automated pen testing is focused on scanning rather than ensuring target environments are as similar as possible to production environments. 

Lastly, automated pen testing tools generally exploit first-level security weaknesses but can’t execute more complex attacks with conditional capabilities where subsequent steps in the kill chain are dependent on the previous one. This is a huge disadvantage of this method, since it limits the ability of pen tests to test all security layers—not ideal, since malware and other malicious tools continue to rapidly increase in sophistication.

Some other weaknesses of this method include:

• Limited to attacks predefined within the tool
• Harder to customize and modify for newer techniques 
• Cannot continuously simulate complex breaches and attacks
• While deeper than traditional vulnerability scanning tools, it is still rather shallow

When Should I Use an Automated Penetration Test?

On-premise environments

As with traditional pen testing, automated pen testing works best in on-premise environments. In these environments, automated pen tests enable a rapid cadence of testing against common external attack vectors programmatically. This ensures security drift is not impacting the external-facing security posture of on-premise infrastructure. 

Automated pen testing is also useful for organizations that wish to test controls on exposed endpoints that have access back into on-premise assets via VPNs. 

When Should I Consider a Different Security Validation Technology?

Cloud and Hybrid Environments

You may be noticing a trend here: the risk with automated pen testing in cloud environments is similar to that of traditional pen testing; however, it may be even less useful in this case. Many automated pen testing tools deploy wide-scale scanning of ports, IPs, PIs, and network endpoints. Cloud companies typically don’t like this and fear poorly operated scanning tools will effectively DDoS unsuspecting customers that are on the same shared services or shared infrastructure—or even in the same data center. As a result, cloud service providers often prohibit cloud pen-testing on shared environments. Those that do only allow limited capabilities for cloud pen-testing tools.

What About Other Security Validation Technologies?

Automated pen testing was purpose built to take traditional pen testing and scale it up. It’s not surprising, then, that it retains many of the same disadvantages. As useful as they can be, automated pen tests simply can’t handle more complex attacks. The next few blogs in this series will cover two methods that are more suited to modern IT environments: 

• Attack surface management
• Breach & attack simulation

If you’d like a full comparison of each approach, take a look at our white paper Six Methods to Test Your Organization’s Resilience to Cyberattacks.