I’m frequently asked about “completeness” when it comes to vulnerability scanning/assessments. “How do you know if all the vulnerabilities have been found?” The short answer is you don’t. Then I proceed to describe the reasons. What got me thinking was why this question is asked so often. I think the answer is only obvious to those experienced in website security. These are faulty assumptions routinely carried over from network vulnerability scanning world that do not apply to webappsec.

In network scanning the list of “well-known” vulnerabilities is large, but also finite. Databases such as OSVDB, SecurityFocus, MITRE (CVE), and others catalog the known universe of issues. Vulnerability coverage by network scanners is likely close to 100%. In “custom” websites the luxury of well-known vulnerabilities or database repositories vanishes. Each new vulnerability identified is more or less a one-off / zero-day issue. Just as with bugs in application code, we truly never know how many vulnerabilities exist in a web bank, e-commerce store, payroll system, or any other custom website. The upper bound in an unknown. Therefore we can never know for sure if any scan/assessment found them all. Vulnerability coverage could be as low as 10-20% or higher in the range of 80-90% or more. The point is we don’t know, its difficult to measure, and changes with each website.

This is a big reason why I’ve been talking a lot about measuring security recently. I’m a big believer in it. Who isn’t? I even took a shot at a Methodology for Comparing website Vulnerability Assessment Solutions. Figured we could use time-it-takes-to-hack-a-website as something we could reliably measure. For some reason I hadn’t got much feedback on the idea. Likely because there hasn’t been customer demand as they’re not REALLY aware of the fact that everything isn’t being found. Whether my methodology works or not, we’re going to need to figure this out. Once customers of ANY webappsec VA solutions gets hacked due to missed vulnerabilities, there’s going to be hell to pay.

Enterprise security professionals have the responsibility of dealing with vulnerabilities. They have to find and fix as many issues as possible wherever they happen to pop up. Varying from one environment to the next, this can be a REALLY big job. To keep up many enlist the help of commercial and open source solutions. The problem is there are perhaps 100’s, or more, vulnerability management/assessment/scan/remediation/consulting vendors all targeting a specific niche of the vulnerability stack in their own special way. It’s a confusing landscape to say the least.In my position I get asked a lot about who covers what, how is it different from the other guy, or how good is it. I do my best to keep track of these things since it’s my business to know and want to give educated answers. I thought it would be helpful to create a couple of graphics that people researching solutions would be able to use. Less confusion = good.

The first graphic is my take on the “vulnerability stack”, the areas of focus for vulnerability scanning/assessment solutions. This is helpful for asking vendors what area they cover. There are probably some areas I missed, but it’s a first draft. If my technical vulnerabilities and business logic flaws terminology is confusing, please see Technology Alone cannot Defeat website Attacks: Understanding Technical vs. Logical Vulnerabilities.




The second graphic is a vulnerability scanning/assessment vendor comparison chart. Here we’re trying to answer the “who covers what question?” and a foundation to ask how they are different. I know some will vendors claim they do more that what the chart indicates, but I’m listing only their main areas of focus. If someone happens to add a website vuln check or two, it doesn’t make them a network scanner. Likewise if website scanners adds a few network checks, it hardly a new Nessus. A decent amount of comprehensiveness in the block is required. Enjoy!

By Jeremiah Grossman
Security researcher Dave Ferguson posted a Netflix CSRF advisory to a few mailing lists. A nice find and responsibly disclosed. Included was the ability for attacker to add movies to your rental queue, change the name and address on your account, etc etc. As Rsnake said, “pretty scary”, and a vulnerability I’ve called the sleeping giant. C-Net’s Joris Evers covered the incident, Netflix fixes Web 2.0 bugs, and described the severity of the problem nicely. There are some parts of the story that require more context. I've met reporter Joris Evers, nice guy, but as sometimes happens, important technical aspects are left out.

“The problems were repaired before they became publicly known, Steve Swasey, a Netflix spokesman, said on Monday.”

It would be interesting to know how much time and effort it took to fix the issue.

“The problems were repaired before they became publicly known, Steve Swasey, a Netflix spokesman, said on Monday. "It is an extremely remote possibility that it would have affected any of Netflix's 5.2 million members," he said.”

For the moment I’d agree about the low percentage probability. The problem is it’s hard to be sure. CSRF is really hard to spot in the logs since the attack looks like normal web server traffic. In fact, even the logged-in user is the one sending the HTTP request, not the hacker. As the criminal element becomes increasingly familiar with CSRF, we can expect increased usage of the exploit.

"Design flaws in the Netflix Web site were the cause of the relatively new type of weakness in websites, …. said Dave Ferguson..."

If "relatively new" means, 6-8 years or more, then OK.

"This type of attack is only suitable for a certain type of Web site. It just happens to be that Netflix is the perfect example," he said. One key thing is when the majority of users keep themselves logged in.”

Not so much. Nearly every website I’ve ever seen has CSRF vulnerabilities. The only difference between one website and another is the features present, which dictate the severity of an exploit. If a website doesn’t do anything, CSRF is meaningless. If users can log-in, change passwords, post comments, etc. then we’re talking about something more damaging.

"Netflix is audited all the time for security," Swasey said. "There was some level of exposure, although not serious." At no point was customer data such as credit card numbers at risk, he stressed.”

With CSRF, credit card numbers are not necessarily the most important piece of data. Neither are usernames and password. When in control of a browser’s authenticated access, a malicious attacker wouldn’t need that data in order to transfer money out of a bank account. Or in this case hi-jack a users Netflix account.

"Cross Site Request Forgery, or XSRF, is seen as one of the security problems that affect feature-rich Web sites. These "Web 2.0" sites often offer an experience more like using a desktop application than like using the Web."

This part and the title of the story reference the ambiguous Web 2.0 moniker. The reality is CSRF affects all websites no matter what technology version they use. AJAX, Flash, its all the same in this respect. For the rest of us, if you think XSS is/was bad and clogs the mailing lists? Just wait till the CSRF issues hit the security scene en mass.

P.S. I’m a Netflix customer, great service and gets my best recommendation!