emen
> >>
> >> <http://www.thetaphi.de/> http://www.thetaphi.de
> >>
> >> eMail: u...@thetaphi.de
> >>
> >>
> >>
> >> From: arun k [mailto:arunk...@gmail.com]
> >> Sent: Wednesday, January 30, 2013 12:04 PM
> >> To: java-user
t;>
>> Uwe
>>
>>
>>
>> -
>>
>> Uwe Schindler
>>
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>
>> <http://www.thetaphi.de/> http://www.thetaphi.de
>>
>> eMail: u...@thetaphi.de
>>
>>
>>
>>
t;
> From: arun k [mailto:arunk...@gmail.com]
> Sent: Wednesday, January 30, 2013 12:04 PM
> To: java-user
> Subject: Re: CompressingStoredFieldsFormat doesn't show improvement
>
>
>
> Adrein,
> Please find the attached profilers report.
>
>
>
> On Wed,
phi.de
From: arun k [mailto:arunk...@gmail.com]
Sent: Wednesday, January 30, 2013 12:04 PM
To: java-user
Subject: Re: CompressingStoredFieldsFormat doesn't show improvement
Adrein,
Please find the attached profilers report.
On Wed, Jan 30, 2013 at 3:35 PM, Adrien Grand wrote:
On
Adrein,
Please find the attached profilers report.
On Wed, Jan 30, 2013 at 3:35 PM, Adrien Grand wrote:
> On Wed, Jan 30, 2013 at 8:08 AM, arun k wrote:
> > Adrein,
> >
> > I have created an index of size 370M of 1 million docs of 40 fields of 40
> > chars and did the profiling.
> > I see that
On Wed, Jan 30, 2013 at 8:08 AM, arun k wrote:
> Adrein,
>
> I have created an index of size 370M of 1 million docs of 40 fields of 40
> chars and did the profiling.
> I see that the indexing and in particular the addDocument &
> ConcurrentMergeScheduler in 4.1 takes double the time compared to 3.
Adrein,
I have created an index of size 370M of 1 million docs of 40 fields of 40
chars and did the profiling.
I see that the indexing and in particular the addDocument &
ConcurrentMergeScheduler in 4.1 takes double the time compared to 3.0.2.
Looks like CompressionStoredFieldsFormat is of little
Arun,
Lucene uses a very light compression algorithm so I'm a little
surprised it can make indexing 2x slower. Could you run indexing under
a profiler to make sure it really is what makes indexing slower?
Thanks!
--
Adrien
-
T
Fair point, but with such small numbers per index any variation
between versions is likely to just be noise. It is also certainly
possible that on your index the compressing format may not help.
An aside: have you considered merging the thousands of small indexes
into one, with some field to iden
Hi Ian,
you r right in that if we have 1 index of say 15 Mb there is no prob but i
have thousands of such indexes.
So the time will add up with the number of such indexes being open
simultaneously and parallel indexing.
Arun
On Tue, Jan 29, 2013 at 7:09 PM, Ian Lea wrote:
> I make that about
I make that about 15Mb of data - trivial. What happens if you make
each field 400 chars and index a million or two? If you really have
that few docs, what are you worrying about?
A doubling of indexing time from 3.0.2 to 4.1 is surprising, but for
40k docs are we talking about it taking 2 second
11 matches
Mail list logo
