I agree - vulnerability assessment tools are only part of the story, and need to be set correctly to imitate real world attacks, such as timed scans, newer BIND attacks, port 80 attacks, etc.
It is equally important to know what the tool is actually doing - for example, does the tool just check the settings for passwords to determine limitation of attempts? If so, it does not check the several different types of login which do not check the limiting file... Also, are you testing the right perspective? from inside or outside the firewall, the switched section, the IDS/IPS portion, etc? It's amazing how many network budgets don't allow sufficient sensors to handle even appropriate, much less complete, placement... I remember checking a network with a tool which did ping sweeps, and finding out it was so saturated with traffic that the ping sweeps brought it down in the middle of business transactions - definitely a vulnerability, but not possible from outside the firewall. Was my test correct? Warm Regards, Dave Druitt -- CSO InfoSec Group 703-626-6516 "using words to describe magic is like using a screwdriver to cut roast beef" -Tom Robbins "there is a big difference betwen kneeling down and bending over" -Bob Dylan -------------- Original message from [EMAIL PROTECTED]: -------------- > Leea, > > > Your post raises an interesting topic, how often do users perform an > assessment > of their key security defenses to prove that they perform operationally as > described in the marketing materials, my bet is not that often in reality. > > > I have worked in the security market space for the last 15 years and during > that > time have seen many end users want to but not really know how to test their > security defenses. You spend a lot of money on these systems, then many times > users put their faith that the product is working as advertised without realy > being able to prove it or having the necessary tools to help. > > > One regular post to this list is "can I use a vulnerability scanner to test > my > IDS/IPS", the answer is generally no as they are not designed for that > purpose. > > > There are a number of things that you should really look at when testing an > IDS > or IPS system and one of the most important things is just how useable is it? > > > If the worst happens and some kind of attack is picked up does the management > console become unusable due to the scale and volume of alerts? (I've seen > many > deployments where a slight burst in activity can make the management system > become a monster and un useable) > > > How easy is it to spot if a sensor has gone off line? (I've seen many > occasions > when acording to the management console the sensor is working fine and active > but in reality somehow it has "gone to sleep" and is not picking up anything. > > > There are a number of resources out there to help you > > > http://www.karalon.com/products.htm > > > The Tolly Group also published a whitepaper on IPS testing and benchmarking > you > may find intresting. > > > http://www.tolly.com/ts/2006/TollyEdge/IPS-Wired/TollyWP206115TollyEdgeIPS-Wired > > -May2006.pdf > > > Regards > > Tony > > > > Tony Haywood > > CTO > > www.karalon.com > > Audit, Test, Prove & Validate > > ------------------------------------------------------------------------ > Test Your IDS > > Is your IDS deployed correctly? > Find out quickly and easily by testing it > with real-world attacks from CORE IMPACT. > Go to > http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_ > > sfw > to learn more. > ------------------------------------------------------------------------ > ------------------------------------------------------------------------ Test Your IDS Is your IDS deployed correctly? Find out quickly and easily by testing it with real-world attacks from CORE IMPACT. Go to http://www.coresecurity.com/index.php5?module=Form&action=impact&campaign=intro_sfw to learn more. ------------------------------------------------------------------------
