Paul Looney wrote:
I have nothing to add directly to the chunk vs array discussion
(Trevor's reply was very good) but I have often found it helpful to
increase the speed of compound selections by breaking them into
individual ones.
For instance if you have a large database of names and sexes and you
want to select every female named "Jan" ("Jan" could be male or female).
Select all of the Jans first (this will run much faster than the
compound selection).
Then select all of the females from the result of the first selection
(this will run faster because it is searching only "Jan"s - a very
small list).
This double selection will run faster than a single compound selection.
Obviously this requires a known data-set where one filter will
eliminate a lot of records (selecting "female", then selecting "Jan"
would be much slower in our example because, presumably, half of the
list is female and a small portion is Jan).
On many lists this can create a much bigger speed difference than
chunk vs array variance you noted.
One of the tough challenges with this sort of benchmarking is that
different methods will favor different test cases.
But with delimited rows and columns, I haven't found a way to make a
two-pass search run faster than one pass, except in very specialized
cases as you noted.
There's a temptation to use the filter command for the first pass, but
filter is only faster when testing the first few items; filtering on the
10th item is much slower, and attempting to test the 50th item in a
sample data set caused Rev to hang. RegEx is a harsh mistress.
In my case, I don't often know in advance which item will be searched.
The queries I'm running usually come from a Search dialog in which the
user can specify criteria. I could make the search function smart
enough to special-case certain types of searches to use a two-pass
method in which the first pass is the filter command where practical,
but the overhead of analyzing both the query and the data to make such
determinations may detract from the benefits of doing so, esp. since my
continued testing on this is increasingly nudging me toward
multi-dimensional arrays anyway. Even with the data bloat and the
surprising overhead of moving arrays in and out of storage, with a
little extra work to deal with those the performance of arrays seems
unbeatable in the broadest range of use cases I've run thus far.
--
Richard Gaskin
Fourth World
Revolution training and consulting: http://www.fourthworld.com
Webzine for Rev developers: http://www.revjournal.com
_______________________________________________
use-revolution mailing list
use-revolution@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-revolution
