Fortunately, not all of the 45000 are used every time, and then not all of them are the list... Anyway, no more worries!! I just had a very intense brainstorming session with our group and we came up with an awsome way to use the SQL LIKE '%CA%' -- this is going to be SUPERFAST. No Java String operation will be able to touch this new SQL, ever.
Thanks to all who pitched in.
:->
Greg
-----Original Message-----
From: Barzilai Spinak [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, March 26, 2003 12:08 PM
To: jdjlist
Subject: [jdjlist] Re: String List Compare
45000, how often? At 15 ms each, it takes like 12 minutes.
If it's only once... you have wasted a lot of time already hehe
If you need to do it once per hour in your system, then it's a different
problem.
Is this a fixed set of 45000 entries? Or is it variable all the time?
If it's variable then converting each entry to whatever (like an array
or Map) before
doing a search probably will introduce a lot of overhead and will defeat
any purpose of
using a faster function.
If it's a fixed set and you have enough RAM then you can try one of the
suggestions to convert
it to a faster data structure once, and then do fast searches for the
rest of your process life....
If it varies but not so often, you can try some kind of cache where the
most recent/frequently
used entries are found first. This speeds things up amazingly in some
cases which I'm not sure it's yours.
Greg Nudelman wrote:
> Nothing. Except I have to do 45000 of these... and the data is quite
> dirty as I metioned, and I am in no hurry to try and clean it up...
> and write the validation/cleanup routine for the new data... You can
> say I am somewhat lazy. hehehehe
>
> I think of all the Java String operations I've tested, indexOf is
> nether the slowest, nor the fastest, clocking in at 15 miliseconds.
> To contrast, String.equalsIgnoreCase is a hog, at 28 miliseconds, and
> String.equals is an acceptable 7 miliseconds, which is still far away
> from the coveted int == int is only 3 miliseconds. (on my test
> strings, your results will vary).
>
> However, I wanted to make sure I don't miss something more obvious.
> For example, SQL LIKE is a great idea, however, my DB data structure
> may unfortunately prevent me from doing this (I have to think some
> more, maybe I can use it). I also like the HashTable lookup idea, but
> java.util.StringTokenizer is so unbelivably slow, that I will have to
> write one of my own if I am to implement this method. And those perl
> utilities are priceless!!
>
> Great ideas, everyone, Thank you so much!!!
>
> Greg
>
> -----Original Message-----
>
---
You are currently subscribed to jdjlist as: [EMAIL PROTECTED]
To unsubscribe send a blank email to [EMAIL PROTECTED]
http://www.sys-con.com/fusetalk
You are currently subscribed to jdjlist as: [EMAIL PROTECTED]
To unsubscribe send a blank email to [EMAIL PROTECTED]
http://www.sys-con.com/fusetalk
