Hi Ira, On 12:38 Thu 21 Jan , Ira Weiny wrote: > > Here is some test data on a real cluster. > > 09:49:10 > ibhosts | wc -l > 1158 > > 09:49:28 > ibswitches | wc -l > 281 > > 09:44:45 > time ./subnet_discover -n 1 > /dev/null > > real 0m1.414s > user 0m0.309s > sys 0m0.244s > > 09:44:55 > time ./subnet_discover -n 2 > /dev/null > > real 0m1.025s > user 0m0.284s > sys 0m0.201s > > 09:45:00 > time ./subnet_discover -n 4 > /dev/null > > real 0m0.644s > user 0m0.268s > sys 0m0.228s > > 09:45:04 > time ./subnet_discover -n 8 > /dev/null > > real 0m0.550s > user 0m0.253s > sys 0m0.184s > > 09:45:08 > time ./subnet_discover -n 12 > /dev/null > > real 0m0.524s > user 0m0.207s > sys 0m0.201s > > 09:45:14 > time ./subnet_discover -n 16 > /dev/null > > real 0m0.432s > user 0m0.248s > sys 0m0.144s
Looks like a very nice results. Thanks for probing this. > 09:45:18 > time ./subnet_discover -n 32 > /dev/null > > real 0m0.484s > user 0m0.260s > sys 0m0.150s With '-n 32' the total time is increased. It is probably due to VL15 packets drops on switches. This can be ensured by checking VL15Drops counter value. Also in this case using different from default timeout values with ./subnet_discover (for example --timeout=1000 (default is 100ms)) will affect the total running time (this doesn't happen when packets were not dropped and ib_mad retrying mechanism was not activated). > 09:45:57 > time ibnetdiscover > /dev/null > > real 0m3.180s > user 0m0.068s > sys 0m0.672s > > > What I find most interesting is that your test utility runs nearly 2x faster > even when there is only 1 outstanding MAD. :-/ ibnetdiscover (libibnetdisc) > does do a lot more with the data but I would not have expected such a > difference. Your fix for subnet_discover explains the difference. > I would appear that your algorithm is superior. I will look at converting > libibnetdisc, test, and submit a patch. Thanks. Sasha -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
