Charles Plessy wrote:
As I had better to spend my time on analysing some data with
my script rather than analysing the script itself, I eventually
swiched to DBI::SQLite, which solved my problem by performing the
query in a few seconds.
There's no question, SQLite is faster than DBD::CSV for most things. If
someone asks me for a recommendation for a database to use and they have large
and or complex data, don't care about the format of the data, and speed is an
issue, I never recommend my own modules :-). I hope you're using mod_perl or
something because connecting to the database is one area where SQLite is
slower. It's also slower for inserts. I'd like to point out also that
SQL::Statement, the underlying engine for DBD::CSV, is undergoing some major
changes and is becoming faster on each benchmark and that the purpose of the
pure perl DBDs like DBD::CSV is to provide access to human readable data,
unconventional datasources, and to provide support for platforms and contexts
where compilation is not an option, not to try to rival the speed of RDBMSs
written in C.
The query was something like :
$query = "SELECT * FROM $db_table WHERE SYMBOL1 LIKE \'%${Search}%\' OR SYMBOL2 LIKE \'%${Search}%\'" ;
LIKE and CLIKE with wildcards are full text searches and are always
going to be slow relative to other kinds of searches.
You may wonder the purpose of this mail, as there is no new
crucial information. The reason is that I do not like not knowing the
end of the story when I browse archived threads.
Thanks, I appreciate hearing back. Good luck!
--
Jeff
