I was wrong, the ratio is "only" 1 to 1174 :-) . But I think substring()
is still not so relevant... ;-) .
Seriously, FYI, the reason for such high ratio is that substring() works
at string level, while toLowerCase() breaks down to a character level
loop, and in my profiling (using a Postage "release" run) the average
token string length was about 70 chars (probably lots of binary data
converted to ascii), as the max token length to be considered is set to
90 chars. Perhaps we could reduce it to a smaller value.
But anyway the involved cpu now has dramatically gone down to very small
values. And it wasn't so dramatic.
Vincenzo
Bernd Fondermann wrote:
wow. :-)
ok, fine.
Bernd
On 8/31/06, Vincenzo Gianferrari Pini
<[EMAIL PROTECTED]> wrote:
The substring cpu time is not relevant compared to toLowerCase: 1 to
60000 ratio :-)
Let's keep it as is.
Vincenzo
Stefano Bagnara wrote:
> Maybe he's referring to "tokenLower.substring(0, end)".
> This appears twice in your code and could be moved to a local
variable.
>
> Stefano
>
> Vincenzo Gianferrari Pini wrote:
>
>> Bernd,
>>
>> I don't understand what you mean by "duplicated substrings".
>>
>> If you mean the substrings added to the tokens ArrayList, only the
>> most significant of them (highest "probability strength") is later on
>> kept by the calling method (getTokenProbabilityStrengths). This is
>> the way it is expected to work.
>>
>> If you have seen something else please let me know.
>>
>> Vincenzo
>>
>>
>> Bernd Fondermann wrote:
>>
>>> Vincenzo,
>>>
>>> do you intend to also eliminate the duplicated substrings or does it
>>> not significantly lower memory/cpu load?
>>>
>>> Bernd
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
