If you are like many CISOs, you feel pressure to increase the value of your security testing budget. And if you are one of the 53% of enterprises reporting stagnant or decreasing budgets in 2024, you have even more work cut out for you.
Increasing testing value requires a re-evaluation of nearly everything. Tackle tool sprawl. Optimize workflows. Reduce false positives. Review cloud spend. All while demonstrating ROI even in the absence of incidents.
This post is about ways to reach these goals. We will then work through an example to provide some color. And at the end I will talk about how CyCognito can help accelerate them.
The good news is that whatever path you take, you will have a stronger, more valuable security testing program with data to back it up.
Let’s look at value first. Traditionally, value is understood as a ratio of benefit to cost. Here we define value as a ratio of tool effectiveness to cost — how much each dollar spent on testing tools helps reduce risk and mitigate threats.
Both terms are expressed in currency. Your decisions must achieve the right balance of cost and effectiveness. Staff health needs to be at the forefront – there is a limit to how much costs can be reduced without creating an unhealthy work environment. The bare minimum is a ratio greater than 1, where “$ effectiveness” exceeds “$ spend.”
In many organizations, reducing hard costs—for example, licensing, compute, storage, and telemetry — are the most straightforward path to increased value. This makes sense – a good movie at $5 a ticket is better value than the same movie at $20 a ticket.
Consider these as a starting point:
A note on labor costs. While labor is technically part of hard costs, for this example we will focus on increasing the efficiency of existing staff, not reducing headcount. Security testing and incident response requires healthy people, or it won’t work no matter what you do.
An average organization manages 76 security tools. Even a third of that number is considerable. Some tools are there because it’s difficult or expensive to replace them. Others because they’ve always been there. Regardless of the reason, if you’re spending money on tools that add marginal value, get rid of them.
Finding tools to remove requires a structured approach:
Then, talk to your testing teams about what is needed. Have them rank tools from 1 to 3, with 3 being critical and 1 being unhelpful. Provide why. They likely have strong opinions about what is working and what isn’t.
Next, let’s look at effectiveness. A tool’s effectiveness is the measurable benefit it provides. The more effective the tool, the higher its value.
Here are ways to increase effectiveness:
The irony is that these improvements can add costs (usually labor) so you have to weigh the benefits against the expenses. Automation should take precedence–a single short term investment in automation pays dividends long term.
Here are some questions to consider when evaluating the effectiveness of tools. Make sure to add numbers to the assessment so that it can be measured.
Security teams have strong opinions about the tools they use. If a tool is effective but difficult or time consuming to use it isn’t as valuable unless it is essential for occasional use. But even then, if it’s expensive, consider alternate tools that can get the job done on an ad-hoc basis.
Assume an organization with several thousand externally-exposed web applications and network services. This organization uses vulnerability scanning, app sec, manual pen testing, security ratings and bug bounties.
Lets list the technologies in a table along with license cost and the number of staff dedicated to it. License costs are examples only and labor costs are simplified to $166K per FTE, fully loaded.
Testing | Total Annual License $ | Dedicated FTEs |
---|---|---|
Vulnerability Scanning with Add-on Modules (e.g. Tenable, Rapid7) | $85K | 3 analysts |
DAST (e.g. Burp Suite Pro, Invicti) | $51K | 1 engineer, 1 architect |
Manual Pen Resting (e.g. Metasploit, Nessus Pro) | $38K | 2 pen testers |
Bug Bounty | $50K | – |
Security Ratings Service | $30K | – |
SUB-TOTALS | $254K | $1,162K |
TOTAL | $1,416K |
Effectiveness is the savings you receive when the security testing tool uncovers a vulnerability early. Detecting a critical/high vulnerability early eliminates associated incident costs. Assumptions for calculating effectiveness include: 20 vulnerabilities per asset annually, 1% of those vulnerabilities are critical/high severity, 20% of those critical vulnerabilities lead to incidents and an average incident cost of $26,000. The formula is (Number of critical/high vulnerabilities) × (20% incidents) × $26,000.
Test Type/Approach for external testing | Total Number of Critical /High Vulnerabilities Discovered | Effectiveness value ($) |
---|---|---|
Vulnerability Scanning 80% coverage, 26X per year |
420 | $2,184K |
DAST 10% coverage, 4X per year |
60 | $312K |
Manual Pen Testing 10% coverage, 4X per year |
100 | $520K |
Bug Bounty 1X per year |
20 | $104K |
Security Ratings Service | 3 | $16K |
SUB TOTAL | $3,136K | |
Incidents from unmanaged or inadequately tested assets | 14 | – $364K |
TOTAL | $2,772K |
Based on this data, the example organization has a security tool effectiveness of $2,772K/$1,416K or $1.96.
In other words, for every dollar spent the example organization is receiving nearly $2 in value based on incident count reduction.
What is the right number? Clearly, higher is better. But there isn’t a single number to pursue. You must create a benchmark value for your organization and use it for comparison after changes. More details are better – for example false positives per technology to represent higher costs from time chasing inaccuracies.
Tip → Use this workflow to illustrate ROI even in the absence of incidents.
CyCognito increases your testing effectiveness by eliminating gaps that lead to incidents. It allows a reduction in costs, including redundant or underperforming tools. And since it is delivered as a fully automated SaaS, your teams spend time on results, not configuration.
With CyCognito:
Interested in calculating the value of your testing program? Use CyCognito’s Cost Savings Calculator to provide an estimate of cost reduction and efficiency gains. It’s an invaluable resource for security leaders and a fast way to kick start your effort.
Then reach out to learn more about why experts recognize CyCognito as a best-of-breed EASM provider, supporting application security testing and complex organizational structures.
Jason Pappalexis has worked in cybersecurity for nearly two decades, holding roles across government security administration, third-party testing, solutions architecture, product management, and technical product marketing.
Download the report now to stay ahead of emerging threats and strengthen your organization’s security posture for 2024.
Download the report to learn about the historical trends behind the emergence of exposure management, how to develop a strategic plan and assemble a team to smoothly transition frameworks, and example tech stacks to consider for your organization.
IT Security teams are faced with stagnant or reduced budgets yet need to increase the value of their security testing programs.
Answer a few questions and receive an instant custom report sharing how you can reduce costs and boost your efficiency with CyCognito.
Discover insights on application security, exposure management and other key topics below.
The definitive guide to attack surface management. Learn everything you need to know to reduce your cyber security risk with attack surface management.
Exposure management is a set of processes which allow organizations to assess the visibility, accessibility, and risk factors of their digital assets.
Vulnerability assessment is the process of identifying, quantifying, and prioritizing vulnerabilities in a system.
Explore CyCognito modules ASM, AST and EI in the resources below.
Scalable, continuous, and comprehensive testing for all external assets, all the time.
CyCognito Automated Security Testing dynamically applies payload-based testing techniques across your entire external attack surface.
CyCognito Exploit Intelligence uses threat intelligence about attackers’ behavior and exploitability for enhanced prioritization.