Arian J. Evans wrote... > The problem I had in the past with benchmarks was the huge degree of > customization in each application I would test. While patterns emerge > that are almost always automatable to some degree, the technologies > almost always require hand care-and-feeding to get them to an > effective place. I think this notion of combining the tools with > qualified users is the true potential power of the SaaS solutions that > are coming to market.
It's a pity that the these dynamic-scanning vendors can't work together to come up with a common approach to at least helping this automation you speak of at least part way along. (Yes, I know. I'm dreaming. ;-) Some ideas that I've had in the past is that they could request and make use of: 1) HTTP access logs from Apache and/or the web / application server. These might be especially useful when the logs are specially configured to also collect POST parameters and then the application's regression tests are run against the application to collect the log data. Most web / app servers support Apache HTTPD style access log format, so parsing shouldn't be too terribly difficult in terms of the # of variations they need to handle. 2) For Java, the web.xml could be used to gather data that might allow some automation, especially wrt discovery of dynamic URLs that otherwise difficult to discover by autoscanning. 3) If Struts or Strut2 is being used, gather info from the Struts validators (forget OTTOMH what the XML files called where this is placed, bot those are what I'm referring to). 4) Define some new custom format to allow the information they need to be independently gathered. Ideally this would be minimally some file format (maybe define a DTD or XSD for some XML format), but their tools could offer some GUI interface as well. Of course, I'm not sure I'd expect to see anything like this in my lifetime. At this point, most of the users of these tools don't even see this as a need to the same degree that Arian and readers of SC-L do and it's not clear how vendors addressing these shortcomings IN A COMMON WAY would help them to compete. More likely, we'll get there from here by evolution and vendors copying ideas from one another. The other significant driver AGAINST this as I see it as many vendors sell "professional services" for specialized consulting on how to do these things manually. That bring in extra $$ into their companies so convincing them to give up their cash cow is a hard sell. And as a purchaser of one of these tools, if you don't have the needed expertise in house (many do, but I'm guessing a lot more don't), it's hard to tell your director that you can't use that $75K piece of shelfware that your security group just bought because they can't figure out how to configure it. Instead, they are more likely to quietly just drop another $10K or so for consulting discretely and hope their director or VP doesn't notice. -kevin -- Kevin W. Wall 614.215.4788 Application Security Team / Qwest IT "The most likely way for the world to be destroyed, most experts agree, is by accident. That's where we come in; we're computer professionals. We cause accidents." -- Nathaniel Borenstein, co-creator of MIME _______________________________________________ Secure Coding mailing list (SC-L) SC-L@securecoding.org List information, subscriptions, etc - http://krvw.com/mailman/listinfo/sc-l List charter available at - http://www.securecoding.org/list/charter.php SC-L is hosted and moderated by KRvW Associates, LLC (http://www.KRvW.com) as a free, non-commercial service to the software security community. _______________________________________________