Never Let a Security Analyst Do a Machine’s Job - TCO calculator

By Jim Wachhaus | September 29, 2021
Share:

You know you need to look at your attack surface the way attackers do and invest in external attack surface management (EASM), right?

Excellent.

Good choice.

Now you’ve got to decide how to make it happen. There are a few options: (1) build your own tools and field your own team, (2) cobble together a number of legacy tools and have your team (“human glue”) manually fill in the integration and orchestration gaps, or (3) buy a solution custom-built for the challenge. I talked about some of the results- and benefits-based considerations by answering these questions in my earlier blog about the Build Vs. Buy decision.

Now let’s dive deeper into why it’s important to think one of the factors you need to consider—cost. It is important to have a good idea of how expensive manual efforts (or solutions requiring a lot of manual effort to maintain) really are when it comes to EASM and compare that with an automated solution like the CyCognito platform. Some of the things to think about may surprise you. As I highlight these sometimes-hidden expenses, bear in mind that some of these are costs you will pay even using some legacy commercial tools.

A good guideline is to share with you what our team has uncovered about the hidden cost of siloed tools. These are the considerations that we used when developing our own Total Cost of Ownership (TCO) calculator as you’ll see as you read on. Or, if you like, instead you can schedule a 1x1 meeting with our team we'll be happy to do the calculations for you in our TCO calculator to showcase how much your existing tools cost, how much a set of tools that give the equivalent benefits of CyCognito would cost, and how cost-effective CyCognito really is.

How Many Things?

Figuring out the Total Cost of Ownership (TCO) of your EASM solution starts with two big questions that are usually the most difficult and most basic:

  1. How many subsidiaries does your company have?
  2. How many assets does your company have?

That’s it.

The first question is pretty easy, if available we just open a U.S. Security and Exchange Commission (SEC) Exhibit 21.1; for a private company. And for most government agencies the information is predictably public. For clarity, I’ll just say that in this context “subsidiary” includes business entities, business units, and any of the other names companies use to talk about entities that are subsidiary to a parent company. Based on what we’ve seen with our prospects and customers, tracking those down can often be tricky because not every organization keeps good records. That’s why it can be helpful to have a way of collecting that information automatically, as our platform does, using natural language processing to help map your attack surface.

The second question is tougher because it begs the question: what do you consider an asset? Most organizations have a fair idea of how many alive IP addresses they have, most of the time, because they scan them monthly with a vuln scanner or NMAP or something similar. They often know the number of web applications they consider important, because they are constantly wrangling with penetration testing firms to test a dozen or so for yearly compliance purposes. And larger organizations are almost always using one or more security ratings services because they are trying to figure out how to reduce their cyber insurance rates, and those services might report potential assets and subsidiaries that the organization would not otherwise know about.

 

Number of Organizations | Subsidiaries | M&A Events per YEAR 25 Subsidiaries
Estimated Number of TOTAL EXTERNAL ASSETS* 150,000 Assets

*EXTERNAL ASSETS = IP addresses (Alive and Inactive), Domains, Certificates, and Web Apps

Figure 1. Screen capture from CyCognito’s External Attack Surface Management
TCO Calculator showing the two questions with sample answers
about the number of subsidiaries and assets.

 

With those basics, we can usually create a pretty good estimate of what we are going to actually find, and we will talk about that later. SPOILER: The CyCognito platform will find unexpected assets, and there MAY be issues on those that your security team had no idea about!

The Smaller Questions

But I digress. After you’ve asked the two big questions, the smaller questions to ask are about current processes, because ultimately we want to compare your current processes for external attack surface management with what the CyCognito platform will do. Questions here are about how often you are doing discovery and attribution of assets to your external attack surface, and what percentage of that attack surface you are actually covering. If you say “100%,” well we have some news...

The next questions are about security testing coverage and cadence. How many external assets are you scanning and how frequently? How many assets are you doing penetration testing on and how frequently? And finally, we look at your processes and tools for prioritization, validation, and reporting on remediation, risk, and security posture improvement.

CURRENT PROCESSES CADENCE COVERAGE COVERAGE #
Asset Discovery, Classification and Business Attribution Monthly 70% 105,000
Vulnerability Scanning Monthly 40% 60,000
Penetration Testing Annual 2% 3,000
High Level Reporting Quarterly 80% 120,000
Figure 2 - Screen capture from CyCognito’s External Attack Surface Management
TCO Calculator showing example estimates of coverage and cadence of current processes.

 

Assume Nothing, errrr, Assume Some Things…

From these few inputs, we’ve found that we can accurately predict things about how an organization is currently managing their external attack surface. The first assumptions are around how long it takes to discover assets and contextualize them, and also what tools are used. As it turns out there are A LOT of tools used for discovery. I did a SANS Webinar in May 2021 about this subject if you are curious.

 

The bottom line takeaway is that organizations spend an average of 1 hour discovering and cataloging every 250 to 1,000 assets, and some assets can be much more difficult to attribute or discover, so take longer than that.

Tools used here are usually inexpensive or open source like Shodan or Censys, or security rating services like BitSight and SecurityScorecard. Optimally, you would want these tools to automatically feed an asset management database, but that level of sophistication is sorely lacking in this area. Maybe we’ll talk more about automation and orchestration for external asset management in another blog…


Security Test All The Things

Screen Shot 2021-09-22 at 9.33.17 PM

The next thing to figure out is the cost to test “all the things” found during discovery. This is different from testing all the things you know about now or even testing the things you think are important with a really expensive penetration test... run once, by folks who wear black hoodies and talk a lot about cryptocurrency. We can use your answers to the “smaller questions” to estimate your current expenditures for vulnerability scanning and penetration testing, both generally focused on alive IP addresses only.

We can put more solid numbers to the assumptions about your total costs for these functions if you have your actual hourly labor rates, but it’s generally a good rule of thumb to know that the minimum cost for basic penetration testing of a single IP is more than $480 each time it’s checked. A vulnerability scan for the same asset is many orders of magnitude less expensive, but still not negligible at $0.10 to $1.00 per asset scan. It's that high because you probably aren’t using your vulnerability scanners every week, or even every month, and whatever results come out will need to be validated by someone. The manual effort to pull together all of these results from the different solutions (pen testing, vulnerability scanning, etc.) is laborious and time consuming, and your team is likely further supplementing that data with threat intelligence data feeds to try to get a better sense of just how to prioritize each of the hundreds of critical CVSS scores (i.e., scores of 10) in your attack surface to decide what your security operations and IT teams will be fixing first.


Finally, Do Something

So, you’ve discovered things. And tested things. And all those tests have uncovered attack vectors, like vulnerabilities, misconfigurations, unattended assets, unused domain space, expired certificates, or a number of other problems that surface in IT systems as they age on the internet, waiting for someone to use them for nefarious activities. It’s time to do something about these issues.

We need to take into account that your team will be prioritizing, validating, and reporting on everything that is found and fixed to provide metrics on a job well done. Tools typically associated with this are IT service management software like ServiceNow or ZenDesk and analytics tools like with Splunk or Kenna. There are a myriad of processes involved in moving from issues identified in the attack surface to actually acting on the problems. The bottom line is that, manually, a critical issue is going to require at least 16 hours to go from an open issue to a validated fix. And that is very optimistic without mentioning the cost of re-testing.

So let’s do a little back-of-the-napkin math. If a $40,000 penetration test requires a month to generate a list of 20 issues on a cluster of 10 web applications, your team is paid at a total loaded cost of $85/hour, and it takes only 16 hours per issue (320 hours) to fix them all, then your TCO for that method of detecting, prioritizing, and validating is $40,000+(320*$85)+$40,000. Or $107,200. Or $5,360 per issue. You might be saying, “But, Jim, you have $40,000 in there twice!” Right, because once you’ve fixed the issue, to really know it’s gone you need to validate the issue isn’t there presumably in the same way you identified it. On the other hand maybe you only run 25% of the previous testing, honed on the fixes...you are still out $77,200 or $3,860 per issue.

And do you have an idea of how many issues you would likely identify per year? How about how many of those issues you need to fix? Our research shows that a well-tended attack surface can have as few as 0.01% assets with critical issues per year, while a more unmanaged attack surface can see something like 5.2%, so the range is quite large. These percentages aren’t large, but if there are 1,000 assets, that is still as many as 52 critical issues per year. And organizations with $1B in revenue and above having over 150,000 assets recognize that any efficiencies they can gain in detecting, prioritizing, fixing, validating, and reporting on these issues (and assets) adds up to real money, which demonstrates the value of an automated solution (like ours) by orders of magnitude.


Do Something, Differently

After all of the math and assumptions we have covered here, no statistics, numbers, or even this graphical comparison will likely change anyone’s pre-held beliefs about external attack surface management. Even though the data strongly indicates that the TCO with CyCognito will be 75% or less than current efforts with a much greater frequency (continuously) and better coverage of your internet-exposed attack surface (coverage much, much closer to 100% than you’re achieving now).

Trying to accomplish manually what CyCognito does would cost you 5-6x more per year than you’re already paying, and would still yield unpredictable results: coordinating teams, tools, and processes becomes an expensive burden and most integrations cannot be easily automated.

 

TCO-blog

Figure 3 - Screen capture from CyCognito’s External Attack Surface Management TCO calculator.

 

If you are intrigued and would like to take the next steps in doing attack surface management differently, please schedule some time and we can walk you through our TCO calculator to uncover the hidden costs of your current attack surface management processes today.

 

About Jim Wachhaus

Jim Wachhaus, Director of Technical Product Marketing, has been in technical roles on cybersecurity products for over two decades and is passionate about the discipline of cyber system defense.

Contact Author:
  • linkedin
  • email
mobile

Start Eliminating Your Shadow Risk

Demo Request