I ran a test on one of Preston's real-world data sets (Weather collection) that had around 40,000 files. I am attaching the results. There are three graphs.
The first shows the time for returning the entire XML for all 40000 files. My index algorithm has huge gains over collection, no matter how much of the data is returned. The second shows how the two algorithms perform as the number of files increases. Both linearly increase, but collection has a much higher slope. The last is just a one-point comparison for returning paths that only exist in only 100 out of the 40000 files. Once again, index has a huge advantage. Steven
index results 40000.xlsx
Description: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
