WhiteHat Security - Complete Website Vulnerability Management Services

I'm frequently asked about “completeness” when it comes to vulnerability scanning/assessments. “How do you know if all the vulnerabilities have been found?” The short answer is you don't. Then I proceed to describe the reasons. What got me thinking was why this question is asked so often. I think the answer is only obvious to those experienced in website security. These are faulty assumptions routinely carried over from network vulnerability scanning world that do not apply to webappsec.

In network scanning the list of “well-known” vulnerabilities is large, but also finite. Databases such as OSVDB, SecurityFocus, MITRE (CVE), and others catalog the known universe of issues. Vulnerability coverage by network scanners is likely close to 100%. In “custom” websites the luxury of well-known vulnerabilities or database repositories vanishes. Each new vulnerability identified is more or less a one-off / zero-day issue. Just as with bugs in application code, we truly never know how many vulnerabilities exist in a web bank, e-commerce store, payroll system, or any other custom website. The upper bound in an unknown. Therefore we can never know for sure if any scan/assessment found them all. Vulnerability coverage could be as low as 10-20% or higher in the range of 80-90% or more. The point is we don't know, its difficult to measure, and changes with each website.

This is a big reason why I've been talking a lot about measuring security recently. I'm a big believer in it. Who isn't? I even took a shot at a Methodology for Comparing website Vulnerability Assessment Solutions. Figured we could use time-it-takes-to-hack-a-website as something we could reliably measure. For some reason I hadn't got much feedback on the idea. Likely because there hasn't been customer demand as they're not REALLY aware of the fact that everything isn't being found. Whether my methodology works or not, we're going to need to figure this out. Once customers of ANY webappsec VA solutions gets hacked due to missed vulnerabilities, there's going to be hell to pay.

Defense-in-depth, a concept which most agree with, is where multiple layers of security are protecting the crown jewels. The idea is should any layer fail, which inevitably happens, you're still protected. Nice. In network security there are firewalls, vulnerability assessment, IDS/IPS, patch and config management, training, encryption, anti-virus, etc. each mitigating some risk. As good as they are we know these traditional solutions they're not perfect and don't help much in webappsec. We need to develop a new set of layers. The problem is we haven't figured out or agreed upon which layers the modern webappsec infrastructure is supposed to have.

It's really important that we do or at least start the dialog about what's working and what's not.

Here's what we know. Security inside the SDLC eliminates flawed code, not all. Vulnerability assessments identify vulnerabilities, and miss some. WAF's and IDS's spot and block attacks, some will pass through. We can train ourselves to be experts in some things, but not everything. Patching and configuration protects from the known, not the unknown. Encryption protects data from prying eyes, not all the time. Sure, these solutions are not perfect, nothing is. That's the point of implementing defense-in-depth. Maximize the strength of the available solutions and mitigate they're weaknesses to protect the organizational assets.

What vulnerabilities (blackbox / whitebox) scanners can and can't find is one of the most important topics in website security. Innovation in this area will inevitably determine the industry-accepted vulnerability assessment methodology. Online business depends on this problem being addressed with the right blend of coverage, ease-of-use, and price. For vendors, it's a battleground for which solutions will ultimately be successful in the market. Competitors who do not adapt and push the technology limits will not be around long. I've seen this coming for a while. To the delight of many and frustration of some I've offered presentations, released articles, and written blog posts.

Since founding WhiteHat Security I've long believed that there was no way a scanner, built by me or someone else, could identify anywhere close to all the vulnerabilities in all websites. For years I had no good way to explain or justify my position. It wasn't until I read a fascinating Dr. Dobbs's article (The Halting Problem) from 2003 that established the basis of my current understanding. To quote:

"None other than Alan Turing proved that a Turing Machine (and, thus, all the computers we're interested in these days) cannot decide whether a given program will halt or run continuously. By extension, no program can detect all errors or run-time faults in another program."

Brilliant! This article instantly sparked my interest in the halting problem, undecidable problem, and a bunch of other mathematical proofs. Taking what I had learned, I later introduced the “technical vulnerabilities” and “business logic flaws” terminology during a 2004 Black Hat conference. I guess people enjoyed the terminology because I frequently see others using it. Loosely described, technical vulnerabilities are those that can be found by scanners and business logic flaws must be found by humans (experts).

What needs to be understood is finding each vulnerability class is not exclusive to a single method of identification. Scanners and humans can in fact identify both technical and logical vulnerabilities. How effective they are is the real question. Observe the following diagram. (Don't take the numbers too literally, the diagram is meant to enforce concepts more than precise measurements.)

1. Scanners are way more adept at finding the majority of technical vulnerabilities. Mostly because the vast number of tests required to be exhaustive is too time consuming for a human (expert).
2. Humans (experts) are much better suited to finding business logic flaws. The issues are highly complex and require contextual understanding, which scanners (computers) lack.
3. Neither scanner nor human will likely or provably reach 100% vulnerability coverage. Software has bugs (vulnerabilities) and that will probably remain the case for a long time to come.

The coverage scales will slide in different directions with each website encountered. A while back I posted some stats on how vulnerabilities are identified here at WhiteHat. Based on 100 websites, here are the findings.

This numbers are neat on a variety of level. As more people dive into website security, inevitably we'll see more measurements, reviews, and statistics released. The cloud of the unknown will lift and the most effective assessment methodology will reveal itself. I welcome this trend as I think I'm on the right track. Brass tacks...

From what I've seen, malicious web attacks typically target websites on a one-by-one basis rather than shotgun blast approach. The bad guys aren't using commercial scanners, performing full-blown assessments, or even open source tools for that matter. Frankly because they don't need to. A web browser and a single vulnerability is all they really need to profit. That's why I've been harping on comprehensiveness and measuring the effectiveness of the assessment methodologies for so long. Finding some of vulnerabilities, some of the time, on some of the websites - ain't going to cut it. You will get hacked this way. We need to find them all, all of the time, and as fast as possible.

My crystal ball (1-3 years):
1) Standalone back box scanners will transfer from the hands of security personnel to those in development and QA – they'll merge with the white box scanners and finally tightly integrate inside of established IDE's.
2) The one-off vulnerability assessment market (professional services) will give way to managed service model, just like they already have in the network VA world.
3) Majority industry consolidation will occur as customers look for singular security vendors that can address the entire vulnerability stack.

Vulnerability StackFriday, November 17, 2006

Enterprise security professionals have the responsibility of dealing with vulnerabilities. They have to find and fix as many issues as possible wherever they happen to pop up. Varying from one environment to the next, this can be a REALLY big job. To keep up many enlist the help of commercial and open source solutions. The problem is there are perhaps 100's, or more, vulnerability management/assessment/scan/remediation/consulting vendors all targeting a specific niche of the vulnerability stack in their own special way. It's a confusing landscape to say the least.In my position I get asked a lot about who covers what, how is it different from the other guy, or how good is it. I do my best to keep track of these things since it's my business to know and want to give educated answers. I thought it would be helpful to create a couple of graphics that people researching solutions would be able to use. Less confusion = good.

The first graphic is my take on the “vulnerability stack”, the areas of focus for vulnerability scanning/assessment solutions. This is helpful for asking vendors what area they cover. There are probably some areas I missed, but it's a first draft. If my technical vulnerabilities and business logic flaws terminology is confusing, please see Technology Alone cannot Defeat website Attacks: Understanding Technical vs. Logical Vulnerabilities.

vulnerability_stack

The second graphic is a vulnerability scanning/assessment vendor comparison chart. Here we're trying to answer the “who covers what question?” and a foundation to ask how they are different. I know some will vendors claim they do more that what the chart indicates, but I'm listing only their main areas of focus. If someone happens to add a website vuln check or two, it doesn't make them a network scanner. Likewise if website scanners adds a few network checks, it hardly a new Nessus. A decent amount of comprehensiveness in the block is required. Enjoy!

101,435,253 to be somewhat exact. The good folks at Netcraft posted their November 2006 Web Server Survey, which added another 3.5 million from the month before. Check this out for comparision, "The first Netcraft survey in August 1995 found 18,957 hosts". Talk about explosive growth.

Previous milestones in the survey were reached in April 1997 (1 million sites), February 2000 (10 million), September 2000 (20 million), July 2001 (30 million), April 2003 (40 million), May 2004 (50 million), March 2005 (60 million), August 2005 (70 million). April 2006 (80 million ) and August 2006 (90 million). vuln_comparison

Then consider if you counted up the sla.ckers.org board and other places listing vulnerablewebsites, disclosure might reach 1,000. I also guessed in Dark Reading's "The Web App Security Gap" that maybe 10,000 websites might be professionally tested for vulnerabilities by a third-party. Could lower or higher, but I don't think I'm that far off. Anyway, that's about 1/10th of a percent of the total number of sites out there. That doesn't even qualify as a drop in the bucket. The reality is we really have no idea how secure of (in)-secure the Web is.

That didn't take long. Someone was probably saving it and begins to confirm my earlier comments, 5 Tips to NOT Get Hacked Online, about Internet Explorer being an attractive target. The PoC IE 6 and 7 hack as described by RSnake says visiting a malicious web page could read data from any other website your browser can see. Hello web bank, hello web mail, hello intranet. The severity appears underrated since its really easy to exploit and the exposure here is fairly high.

Supposedly this vulnerability was known in IE6 months ago and somehow made it into IE7. Odd. Personally, I think IE7 vulnerabilities are of limited overall risk while the user-base remains small. Several months from now it'll be a different story when migration is in full swing. As security researchers and hardcore fraudsters become familiar with the product internals the risk profile will change. The problem is while IE7 is probably far more secure than its predecessor, less bugs = good, this does not necessarily mean less risky for users.

Security researcher Dave Ferguson posted a Netflix CSRF advisory to a few mailing lists. A nice find and responsibly disclosed. Included was the ability for attacker to add movies to your rental queue, change the name and address on your account, etc etc. As Rsnake said, “pretty scary”, and a vulnerability I've called the sleeping giant. C-Net's Joris Evers covered the incident, Netflix fixes Web 2.0 bugs, and described the severity of the problem nicely. There are some parts of the story that require more context. I've met reporter Joris Evers, nice guy, but as sometimes happens, important technical aspects are left out.

“The problems were repaired before they became publicly known, Steve Swasey, a Netflix spokesman, said on Monday.”

It would be interesting to know how much time and effort it took to fix the issue.

“The problems were repaired before they became publicly known, Steve Swasey, a Netflix spokesman, said on Monday. "It is an extremely remote possibility that it would have affected any of Netflix's 5.2 million members," he said.”

For the moment I'd agree about the low percentage probability. The problem is it's hard to be sure. CSRF is really hard to spot in the logs since the attack looks like normal web server traffic. In fact, even the logged-in user is the one sending the HTTP request, not the hacker. As the criminal element becomes increasingly familiar with CSRF, we can expect increased usage of the exploit.

"Design flaws in the Netflix Web site were the cause of the relatively new type of weakness in websites, …. said Dave Ferguson..."

If "relatively new" means, 6-8 years or more, then OK.

"This type of attack is only suitable for a certain type of Web site. It just happens to be that Netflix is the perfect example," he said. One key thing is when the majority of users keep themselves logged in.”

Not so much. Nearly every website I've ever seen has CSRF vulnerabilities. The only difference between one website and another is the features present, which dictate the severity of an exploit. If a website doesn't do anything, CSRF is meaningless. If users can log-in, change passwords, post comments, etc. then we're talking about something more damaging.

"Netflix is audited all the time for security," Swasey said. "There was some level of exposure, although not serious." At no point was customer data such as credit card numbers at risk, he stressed.”

With CSRF, credit card numbers are not necessarily the most important piece of data. Neither are usernames and password. When in control of a browser's authenticated access, a malicious attacker wouldn't need that data in order to transfer money out of a bank account. Or in this case hi-jack a users Netflix account.

"Cross Site Request Forgery, or XSRF, is seen as one of the security problems that affect feature-rich Web sites. These "Web 2.0" sites often offer an experience more like using a desktop application than like using the Web."

This part and the title of the story reference the ambiguous Web 2.0 moniker. The reality is CSRF affects all websites no matter what technology version they use. AJAX, Flash, its all the same in this respect. For the rest of us, if you think XSS is/was bad and clogs the mailing lists? Just wait till the CSRF issues hit the security scene en mass.

P.S. I'm a Netflix customer, great service and gets my best recommendation!