The most qualified people would probably be the developer themselves. Let them all agree to some metrics, and then conduct testing with each tool in their own environments.

Even with this testing it will prove little… because the idea of comparing them in the first place is pointless. Anyways… now I’m starting to rant, so I’ll just leave it at this.

Stop arguing and waste time… do something to improve the situation instead.

I am presuming you don’t have much Pen testing experience. I can’t remember the last time I have ran a default low port scan and trusted the output enought to declare the box secure in my report, without using other tools with a variety of diferent options, espeically when scanning a host over the internet.

I think your test is very unrealistic and is typical of a ‘Skiript Kiddie’ port scan or a scan of someone who does not need 100% accurate and complete information – try runing mnap against multiple hosts on multiple networks behind any top brand IDS – and then try running unicornscan against the same hosts sitting behind the same IDS.

You would come out with a clear winner and it would not neccesarily be Nmap.

I would also take accuracy as well… but within limits… There are times when I just want the scan done, and in those places… I’ll take the speed with reasonably accurate results. Also avoiding an IDS/IPS has nothing to do with straight scanning a host which is what I was looking at.

I think you would be quite surprised at how many people, both network admins and ‘security people’, don’t learn all the finer points of the tool. It has nothing to do with being in the wrong trade. It has to do with requirements and how many intricacies some of these tools have. I was looking at straight point and shoot and I’ve seen plenty of people in plenty of situations (most situations in fact) do just that. I would argue that the kid is school is more likely the explore the options than a network admin. I don’t know where you work, but in a lot of places you don’t have time to sit and fiddle and learn how to tweak a tool… You especially can’t justify it when there’s another tool that will do just as good of a job straight out of the box… which is one of the problems I have with UnicornScan.

If a network admin or a ‘security person’ as you put it can’t find time to learn how to use a port scanner then they are in the wrong trade – likewise any professional who is using a port scanner in such a way that he may need to use a certain scanner for a certain task will not be using default settings for default scans……a kid in school may do so, but not someone who needs it for their job.

UnicornScan in certain situations massively exceeds Nmap – and vice versa – I can’t speak for Port Bunny as I have never used it.- it all depends on your goal and how you need to achieve it.

I’ll admit that I’m not overly experienced with Unicornscan… That being said… One aspect of a scanner that has to be considered is usability, since most network / security people don’t have time to figure out the settings available for the scanners and learn how to best use them in different situations… which was why I did a default scan for each, which was directly out of the unicornscan documentation. I added the default options for nmap only because it is, as far as I know, the only scanner that performs retries by default and that is somewhat throttled. (Other than unicornscan being limited to 300pps).

Personally, I wanted to see what would happen if I ran the scans in default modes… in the end… I think any scanner will be hard pressed to beat nmap… but that being said… I invite the authors of each of the three scanners to provide me with what they consider to be the “ultimate” scan settings for their scanners. I’ll perform additional scans with each provided setting and provide an updated chart with the data from the “tuned” scans.

As an example… I can shave 11 seconds of of nmap’s 14 second default scan time against the vista box, simply by providing the -sS flag, which I probably should have done but I wanted to run everything “out of the box”

By default, scanning all 65k ports should take ~3 minutes, 45 seconds at 300 pps. 300 pps is the default speed that unicornscan goes unless you request a faster speed with the -r flag.

If it was only taking 9 seconds, and you were not requesting a higher speed, then I’d guess that the TSC on your computer isn’t functioning properly. If the TSC isn’t working properly, all of your scans will suffer from dropped packets. Try again with -d2 (gtod timer).

Nmap, portbunny, and unicornscan have very different methods of accomplishing what they’re doing. I could run “heads up tests” that show each to be the clear winner, depending on what metrics I choose to measure. This particular comparison wasn’t an accurate representation of any of the projects.

–Robert