subject:"Memory usage"

Re: Question about startup memory usage

2019-11-14 Thread Shawn Heisey


On 11/14/2019 1:46 AM, Hongxu Ma wrote:

Thank you @Shawn Heisey , you help me many times.

My -xms=1G
When restart solr, I can see the progress of memory increasing (from 1G to 9G, 
took near 10s).

I have a guess: maybe solr is loading some needed files into heap memory, e.g. 
*.tip : term index file. What's your thoughts?


Solr's basic operation involves quite a lot of Java memory allocation. 
Most of what gets allocated turns into garbage almost immediately, but 
Java does not reuse that memory right away ... it can only be reused 
after garbage collection on the appropriate memory region runs.


The algorithms in Java that decide between either grabbing more memory 
(up to the configured heap limit) or running garbage collection are 
beyond my understanding.  For programs with heavy memory allocation, 
like Solr, the preference does seem to lean towards allocating more 
memory if it's available than performing garbage collection.


I can imagine that initial loading of indexes containing billions of 
documents will require quite a bit of heap.  I do not know what data is 
stored in that memory.


Thanks,
Shawn

Re: Question about startup memory usage

2019-11-14 Thread Hongxu Ma

Thank you @Shawn Heisey<mailto:apa...@elyograg.org> , you help me many times.

My -xms=1G
When restart solr, I can see the progress of memory increasing (from 1G to 9G, 
took near 10s).

I have a guess: maybe solr is loading some needed files into heap memory, e.g. 
*.tip : term index file. What's your thoughts?

thanks.

From: Shawn Heisey 
Sent: Thursday, November 14, 2019 1:15
To: solr-user@lucene.apache.org 
Subject: Re: Question about startup memory usage

On 11/13/2019 2:03 AM, Hongxu Ma wrote:
> I have a solr-cloud cluster with a big collection, after startup (no any 
> search/index operations), its jvm memory usage is 9GB (via top: RES).
>
> Cluster and collection info:
> each host: total 64G mem, two solr nodes with -xmx=15G
> collection: total 9B billion docs (but each doc is very small: only some 
> bytes), total size 3TB.
>
> My question is:
> Is the 9G mem usage after startup normal? If so, I am worried that the follow 
> up index/search operations will cause an OOM error.
> And how can I reduce the memory usage? Maybe I should introduce more host 
> with nodes, but besides this, is there any other solution?

With the "-Xmx=15G" option, you've told Java that it can use up to 15GB
for heap.  It's total resident memory usage is eventually going to reach
a little over 15GB and probably never go down.  This is how Java works.

The amount of memory that Java allocates immediately on program startup
is related to the -Xms setting.  Normally Solr uses the same number for
both -Xms and -Xmx, but that can be changed if you desire.  We recommend
using the same number.  If -Xms is smaller than -Xmx, Java may allocate
less memory as soon as it starts, then Solr is going to run through its
startup procedure.  We will not know exactly how much memory allocation
is going to occur when that happens ... but with billions of documents,
it's not going to be small.

Thanks,
Shawn

Re: Question about startup memory usage

2019-11-13 Thread Shawn Heisey


On 11/13/2019 2:03 AM, Hongxu Ma wrote:

I have a solr-cloud cluster with a big collection, after startup (no any 
search/index operations), its jvm memory usage is 9GB (via top: RES).

Cluster and collection info:
each host: total 64G mem, two solr nodes with -xmx=15G
collection: total 9B billion docs (but each doc is very small: only some 
bytes), total size 3TB.

My question is:
Is the 9G mem usage after startup normal? If so, I am worried that the follow 
up index/search operations will cause an OOM error.
And how can I reduce the memory usage? Maybe I should introduce more host with 
nodes, but besides this, is there any other solution?


With the "-Xmx=15G" option, you've told Java that it can use up to 15GB 
for heap.  It's total resident memory usage is eventually going to reach 
a little over 15GB and probably never go down.  This is how Java works.


The amount of memory that Java allocates immediately on program startup 
is related to the -Xms setting.  Normally Solr uses the same number for 
both -Xms and -Xmx, but that can be changed if you desire.  We recommend 
using the same number.  If -Xms is smaller than -Xmx, Java may allocate 
less memory as soon as it starts, then Solr is going to run through its 
startup procedure.  We will not know exactly how much memory allocation 
is going to occur when that happens ... but with billions of documents, 
it's not going to be small.


Thanks,
Shawn

Question about startup memory usage

2019-11-13 Thread Hongxu Ma

Hi
I have a solr-cloud cluster with a big collection, after startup (no any 
search/index operations), its jvm memory usage is 9GB (via top: RES).

Cluster and collection info:
each host: total 64G mem, two solr nodes with -xmx=15G
collection: total 9B billion docs (but each doc is very small: only some 
bytes), total size 3TB.

My question is:
Is the 9G mem usage after startup normal? If so, I am worried that the follow 
up index/search operations will cause an OOM error.
And how can I reduce the memory usage? Maybe I should introduce more host with 
nodes, but besides this, is there any other solution?

Thanks.

Re: Question about memory usage and file handling

2019-11-11 Thread Erick Erickson

(1) no. The internal Ram buffer will pretty much limit the amount of heap used 
however.

(2) You actually have several segments. “.cfs” stands for “Compound File”, see: 

https://lucene.apache.org/core/7_1_0/core/org/apache/lucene/codecs/lucene70/package-summary.html
"An optional "virtual" file consisting of all the other index files for systems 
that frequently run out of file handles.”

IOW, _0.cfs is a complete segment. _1.cfs is a different, complete segment etc. 
The merge policy (TieredMergePolicy) controls when these are used .vs. the 
segment being kept in separate files.

New segments are created whenever the ram buffer is flushed or whenever you do 
a commit (closing the IW also creates a segment IIUC). However, under control 
of the merge policy, segments are merged. See: 
http://blog.mikemccandless.com/2011/02/visualizing-lucenes-segment-merges.html

You’re confusing closing a writer with merging segments. Essentially, every 
time a commit happens, the merge policy is called to determine if segments 
should be merged, see Mike’s blog above.

Additionally, you say "I was hoping there would be only _0.cfs file”. This’ll 
pretty much never happen. Segment names always increase, at best you’d have 
something like _ab.cfs, if not 10-15 _ab* files.

Lucene likes file handles, essentially when searching a file handle will be 
open for _every_ file in your index all the time.

All that said, counting the number of files seems like a waste of time. If 
you’re running on a *nix box, the usual (Solr I’ll admit, but I think it 
applies to Lucene as well) is to set the limit to 65K or so.

And if you’re truly concerned, and since you say this is an immutable, you can 
do a forceMerge. Prior to Lucene 7.5, the would by default form exactly one 
segment. For Lucene 7.5 and later, it’ll respect max segment size (a parameter 
in TMP, defaults to 5g) unless you specify a segment count of 1.

Best,
Erick

> On Nov 11, 2019, at 5:47 PM, Shawn Heisey  wrote:
> 
> On 11/11/2019 1:40 PM, siddharth teotia wrote:
>> I have a few questions about Lucene indexing and file handling. It would be
>> great if someone can help with these. I had earlier asked these questions
>> on gene...@lucene.apache.org but was asked to seek help here.
> 
> This mailing list (solr-user) is for Solr.  Questions about Lucene do not 
> belong on this list.
> 
> You should ask on the java-user mailing list, which is for questions related 
> to the core (Java) version of Lucene.
> 
> http://lucene.apache.org/core/discussion.html#java-user-list-java-userluceneapacheorg
> 
> I have put the original sender address in the BCC field just in case you are 
> not subscribed here.
> 
> Thanks,
> Shawn

Re: Question about memory usage and file handling

2019-11-11 Thread Shawn Heisey


On 11/11/2019 1:40 PM, siddharth teotia wrote:

I have a few questions about Lucene indexing and file handling. It would be
great if someone can help with these. I had earlier asked these questions
on gene...@lucene.apache.org but was asked to seek help here.


This mailing list (solr-user) is for Solr.  Questions about Lucene do 
not belong on this list.


You should ask on the java-user mailing list, which is for questions 
related to the core (Java) version of Lucene.


http://lucene.apache.org/core/discussion.html#java-user-list-java-userluceneapacheorg

I have put the original sender address in the BCC field just in case you 
are not subscribed here.


Thanks,
Shawn

Question about memory usage and file handling

2019-11-11 Thread siddharth teotia

Hi All,

I have a few questions about Lucene indexing and file handling. It would be
great if someone can help with these. I had earlier asked these questions
on gene...@lucene.apache.org but was asked to seek help here.


(1) During indexing, is there any knob to tell the writer to use off-heap
for buffering. I didn't find anything in the docs so probably the answer is
no. Just confirming.

(2) I did some experiments with buffering threshold using
setMaxRAMBufferSizeMB() on IndexWriterConfig. I varied it from 16MB
(default), 128MB, 256MB and 512MB. The experiment was ingesting 5million
documents. It turns out that buffering threshold also controls the number
of files that are created in the index directory. In all the cases, I see
only 1 segment (since there was just one segments_1) file but there were
multiple .cfs files  -- _0.cfs, _1.cfs, _2.cfs, _3.cfs.

How can there be multiple cfs files when there is just one segment? My
understanding from the documentation was that all files for each segment
will have the same name but different extension. In this case, even though
there is only 1 segment, there are still cfs files. Does each flush result
in a new file?

The reason to do this experiment is to understand the number of open files
both while building the index and querying. I am not quite sure why I am
seeing multiple CFS files when there is only 1 segment. I was hoping there
would be only_0.cfs file.  This is true when buffer threshold is 512MB, but
there are 2 cfs files when threshold is set to 256MB, 5 cfs files when set
to 128MB and I didn't see the CFS file for the default 16MB threshold.
There were individual files (.fdx, .fdt, .tip etc). I thought by default
Lucene creates a compound file at least after the writer closes. Is that
not true?

I can see that during querying, only the cfs file is kept opened. But I
would like to understand a little bit about the number of cfs files and
based on that we can set the buffering threshold to control the heap
overhead while building the index.

(2) In my experiments, the writer commits and is closed after ingesting all
the 5million documents and after that there is no need for us to index
more. So essentially it is an immutable index. However, I want to
understand the threshold for creating a new segment. Is that pretty high?
Or if the writer is reopened, then the next set of documents will go into
the next segment and so on?

I would really appreciate some help with above questions.

Thanks,
Siddharth

Re: investigating high heap memory usage particularly on overseer / collection leaders

2019-10-22 Thread Paras Lehana

Since you say that it could be related to client usage patterns, have you
tried analyzing queries taking the maximum times? Refer this
<https://lucene.apache.org/solr/guide/7_6/configuring-logging.html#logging-slow-queries>
.

On Tue, 8 Oct 2019 at 02:42, dshih  wrote:

> 3-node SOLR 7.4.0
> 24gb max heap memory
> 13 collections, each with 500mb-2gb index (on disk)
>
> We are investigating high heap memory usage/spikes with our SOLR cluster
> (details above).  After rebooting the cluster, all three instances stay
> under 2gb for about a day.  Then suddenly, one instance (srch01 in the
> below
> graph) spikes to about 7.5gb and begins a cycle of 3gb-7.5gb
> ups-and-downs.
> On this cluster, srch01 is both the overseer and the leader for all
> collections.  A few days later, the same trend begins occurring for another
> node (srch02).
>
> Are there known usage patterns that would cause this kind of memory usage
> with SOLR?  In particular, it seems odd that it would only affect the
> overseer/leaders node for days.  Also, any tips on investigation?  We
> haven't been able to deduce much from visualvm profiling.
>
> Additional context.  For years, we set max heap memory to 4gb.  But our
> SOLR
> instances recently began to OOM.  Increasing to 8gb helped, but the OOMs
> still eventually occurred.  This is how we eventually set it to 24gb
> (following SOLR documentation saying 10-20gb was not uncommon for
> production
> instances).  But the recent change is what makes us suspicious that some
> client usage pattern is the root cause.
>
> <https://lucene.472066.n3.nabble.com/file/t494250/ue1_10-4_to_10-7.jpg>
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
-- 
Regards,

*Paras Lehana* [65871]
Software Programmer, Auto-Suggest,
IndiaMART Intermesh Ltd.

8th Floor, Tower A, Advant-Navis Business Park, Sector 142,
Noida, UP, IN - 201303

Mob.: +91-9560911996
Work: 01203916600 | Extn:  *8173*

-- 
IMPORTANT: 
NEVER share your IndiaMART OTP/ Password with anyone.

investigating high heap memory usage particularly on overseer / collection leaders

2019-10-07 Thread dshih

3-node SOLR 7.4.0
24gb max heap memory
13 collections, each with 500mb-2gb index (on disk)

We are investigating high heap memory usage/spikes with our SOLR cluster
(details above).  After rebooting the cluster, all three instances stay
under 2gb for about a day.  Then suddenly, one instance (srch01 in the below
graph) spikes to about 7.5gb and begins a cycle of 3gb-7.5gb ups-and-downs. 
On this cluster, srch01 is both the overseer and the leader for all
collections.  A few days later, the same trend begins occurring for another
node (srch02).

Are there known usage patterns that would cause this kind of memory usage
with SOLR?  In particular, it seems odd that it would only affect the
overseer/leaders node for days.  Also, any tips on investigation?  We
haven't been able to deduce much from visualvm profiling.

Additional context.  For years, we set max heap memory to 4gb.  But our SOLR
instances recently began to OOM.  Increasing to 8gb helped, but the OOMs
still eventually occurred.  This is how we eventually set it to 24gb
(following SOLR documentation saying 10-20gb was not uncommon for production
instances).  But the recent change is what makes us suspicious that some
client usage pattern is the root cause.

<https://lucene.472066.n3.nabble.com/file/t494250/ue1_10-4_to_10-7.jpg> 



--
Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Synonym filters memory usage

2019-10-02 Thread Dominique Bejean

Thank you for all your responses.
Dominique

Le lun. 30 sept. 2019 à 13:38, Erick Erickson  a
écrit :

> Solr/Lucene _better_ not have a copy of the synonym map for every segment,
> if so it’s a JIRA for sure. I’ve seen indexes with 100s of segments. With a
> large synonym file it’d be terrible.
>
> I would be really, really, really surprised if this is the case. The
> Lucene people are very careful with memory usage and would hop on this in
> an instant if true I’d guess.
>
> Best,
> Erick
>
> > On Sep 30, 2019, at 5:27 AM, Andrea Gazzarini 
> wrote:
> >
> > That sounds really strange to me.
> > Segments are created gradually depending on changes applied to the
> index, while the Schema should have a completely different lifecycle,
> independent from that.
> > If that is true, that would mean each time a new segment is created Solr
> would instantiate a new Schema instance (or at least, assuming this is
> valid only for synonyms, one SynonymFilterFactory, one SynonymFilter, one
> SynonymMap), which again, sounds really strange.
> >
> > Thanks for the point, I'll check and I'll let you know
> >
> > Cheers,
> > Andrea
> >
> > On 30/09/2019 09:58, Bernd Fehling wrote:
> >> Yes, I think so.
> >> While integrating a Thesaurus as synonyms.txt I saw massive memory
> usage.
> >> A heap dump and analysis with MemoryAnalyzer pointed out that the
> >> SynonymMap took 3 times a huge amount of memory, together with each
> >> opened index segment.
> >> Just try it and check that by yourself with heap dump and
> MemoryAnalyzer.
> >>
> >> Regards
> >> Bernd
> >>
> >>
> >> Am 30.09.19 um 09:44 schrieb Andrea Gazzarini:
> >>> mmm, ok for the core but are you sure things in this case are working
> per-segment? I would expect a FilterFactory instance per index, initialized
> at schema loading time.
> >>>
> >>> On 30/09/2019 09:04, Bernd Fehling wrote:
> >>>> And I think this is per core per index segment.
> >>>>
> >>>> 2 cores per instance, each core with 3 index segments, sums up to 6
> times
> >>>> the 2 SynonymMaps. Results in 12 times SynonymMaps.
> >>>>
> >>>> Regards
> >>>> Bernd
> >>>>
> >>>>
> >>>> Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:
> >>>>>   Hi,
> >>>>> looking at the stateful nature of SynonymGraphFilter/FilterFactory
> classes,
> >>>>> the answer should be 2 times (one time per type instance).
> >>>>> The SynonymMap, which internally holds the synonyms table, is a
> private
> >>>>> member of the filter factory and it is loaded each time the factory
> needs
> >>>>> to create a type.
> >>>>>
> >>>>> Best,
> >>>>> Andrea
> >>>>>
> >>>>> On 29/09/2019 23:49, Dominique Bejean wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> My concern is about memory used by synonym filter, especially if
> synonyms
> >>>>> resources files are large.
> >>>>>
> >>>>> If in my schema, there are two field types "TypeSyno1" and
> "TypeSyno2"
> >>>>> using synonym filter with the same synonyms files.
> >>>>> For each of these two field types there are two fields
> >>>>>
> >>>>> Field1 type is TypeSyno1
> >>>>> Field2 type is TypeSyno1
> >>>>> Field3 type is TypeSyno2
> >>>>> Field4 type is TypeSyno2
> >>>>>
> >>>>> How many times is the synonym file loaded in memory ?
> >>>>> 4 times, so one time per field ?
> >>>>> 2 times, so one time per instanciated type ?
> >>>>>
> >>>>> Regards
> >>>>>
> >>>>> Dominique
> >>>
> >
> > --
> > Andrea Gazzarini
> > Search Consultant, R&D Software Engineer
> >
> >
> >
> > mobile: +39 349 513 86 25
> > email: a.gazzar...@sease.io
> >
>
>

Re: Synonym filters memory usage

2019-09-30 Thread Erick Erickson

Solr/Lucene _better_ not have a copy of the synonym map for every segment, if 
so it’s a JIRA for sure. I’ve seen indexes with 100s of segments. With a large 
synonym file it’d be terrible.

I would be really, really, really surprised if this is the case. The Lucene 
people are very careful with memory usage and would hop on this in an instant 
if true I’d guess.

Best,
Erick

> On Sep 30, 2019, at 5:27 AM, Andrea Gazzarini  wrote:
> 
> That sounds really strange to me. 
> Segments are created gradually depending on changes applied to the index, 
> while the Schema should have a completely different lifecycle, independent 
> from that.
> If that is true, that would mean each time a new segment is created Solr 
> would instantiate a new Schema instance (or at least, assuming this is valid 
> only for synonyms, one SynonymFilterFactory, one SynonymFilter, one 
> SynonymMap), which again, sounds really strange.
> 
> Thanks for the point, I'll check and I'll let you know
> 
> Cheers, 
> Andrea
> 
> On 30/09/2019 09:58, Bernd Fehling wrote:
>> Yes, I think so. 
>> While integrating a Thesaurus as synonyms.txt I saw massive memory usage. 
>> A heap dump and analysis with MemoryAnalyzer pointed out that the 
>> SynonymMap took 3 times a huge amount of memory, together with each 
>> opened index segment. 
>> Just try it and check that by yourself with heap dump and MemoryAnalyzer. 
>> 
>> Regards 
>> Bernd 
>> 
>> 
>> Am 30.09.19 um 09:44 schrieb Andrea Gazzarini: 
>>> mmm, ok for the core but are you sure things in this case are working 
>>> per-segment? I would expect a FilterFactory instance per index, initialized 
>>> at schema loading time. 
>>> 
>>> On 30/09/2019 09:04, Bernd Fehling wrote: 
>>>> And I think this is per core per index segment. 
>>>> 
>>>> 2 cores per instance, each core with 3 index segments, sums up to 6 times 
>>>> the 2 SynonymMaps. Results in 12 times SynonymMaps. 
>>>> 
>>>> Regards 
>>>> Bernd 
>>>> 
>>>> 
>>>> Am 30.09.19 um 08:41 schrieb Andrea Gazzarini: 
>>>>>   Hi, 
>>>>> looking at the stateful nature of SynonymGraphFilter/FilterFactory 
>>>>> classes, 
>>>>> the answer should be 2 times (one time per type instance). 
>>>>> The SynonymMap, which internally holds the synonyms table, is a private 
>>>>> member of the filter factory and it is loaded each time the factory needs 
>>>>> to create a type. 
>>>>> 
>>>>> Best, 
>>>>> Andrea 
>>>>> 
>>>>> On 29/09/2019 23:49, Dominique Bejean wrote: 
>>>>> 
>>>>> Hi, 
>>>>> 
>>>>> My concern is about memory used by synonym filter, especially if synonyms 
>>>>> resources files are large. 
>>>>> 
>>>>> If in my schema, there are two field types "TypeSyno1" and "TypeSyno2" 
>>>>> using synonym filter with the same synonyms files. 
>>>>> For each of these two field types there are two fields 
>>>>> 
>>>>> Field1 type is TypeSyno1 
>>>>> Field2 type is TypeSyno1 
>>>>> Field3 type is TypeSyno2 
>>>>> Field4 type is TypeSyno2 
>>>>> 
>>>>> How many times is the synonym file loaded in memory ? 
>>>>> 4 times, so one time per field ? 
>>>>> 2 times, so one time per instanciated type ? 
>>>>> 
>>>>> Regards 
>>>>> 
>>>>> Dominique 
>>> 
> 
> -- 
> Andrea Gazzarini
> Search Consultant, R&D Software Engineer
> 
> 
> 
> mobile: +39 349 513 86 25
> email: a.gazzar...@sease.io 
>

Re: Synonym filters memory usage

2019-09-30 Thread Andrea Gazzarini


That sounds really strange to me.
Segments are created gradually depending on changes applied to the 
index, while the Schema should have a completely different lifecycle, 
independent from that.
If that is true, that would mean each time a new segment is created Solr 
would instantiate a new Schema instance (or at least, assuming this is 
valid only for synonyms, one SynonymFilterFactory, one SynonymFilter, 
one SynonymMap), which again, sounds really strange.


Thanks for the point, I'll check and I'll let you know

Cheers,
Andrea

On 30/09/2019 09:58, Bernd Fehling wrote:

Yes, I think so.
While integrating a Thesaurus as synonyms.txt I saw massive memory usage.
A heap dump and analysis with MemoryAnalyzer pointed out that the
SynonymMap took 3 times a huge amount of memory, together with each
opened index segment.
Just try it and check that by yourself with heap dump and MemoryAnalyzer.

Regards
Bernd


Am 30.09.19 um 09:44 schrieb Andrea Gazzarini:
mmm, ok for the core but are you sure things in this case are working 
per-segment? I would expect a FilterFactory instance per index, 
initialized at schema loading time.


On 30/09/2019 09:04, Bernd Fehling wrote:

And I think this is per core per index segment.

2 cores per instance, each core with 3 index segments, sums up to 6 
times

the 2 SynonymMaps. Results in 12 times SynonymMaps.

Regards
Bernd


Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:

  Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory 
classes,

the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a 
private
member of the filter factory and it is loaded each time the factory 
needs

to create a type.

Best,
Andrea

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if 
synonyms

resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

Field1 type is TypeSyno1
Field2 type is TypeSyno1
Field3 type is TypeSyno2
Field4 type is TypeSyno2

How many times is the synonym file loaded in memory ?
4 times, so one time per field ?
2 times, so one time per instanciated type ?

Regards

Dominique




--
Andrea Gazzarini
/Search Consultant, R&D Software Engineer/

Sease Ltd

mobile: +39 349 513 86 25
email: a.gazzar...@sease.io

Re: Synonym filters memory usage

2019-09-30 Thread Bernd Fehling

Yes, I think so.
While integrating a Thesaurus as synonyms.txt I saw massive memory usage.
A heap dump and analysis with MemoryAnalyzer pointed out that the
SynonymMap took 3 times a huge amount of memory, together with each
opened index segment.
Just try it and check that by yourself with heap dump and MemoryAnalyzer.

Regards
Bernd

Am 30.09.19 um 09:44 schrieb Andrea Gazzarini:
mmm, ok for the core but are you sure things in this case are working per-segment? I would expect a FilterFactory instance per index,
initialized at schema loading time.

On 30/09/2019 09:04, Bernd Fehling wrote:

And I think this is per core per index segment.

2 cores per instance, each core with 3 index segments, sums up to 6 times
the 2 SynonymMaps. Results in 12 times SynonymMaps.

Regards
Bernd

Am 30.09.19 um 08:41 schrieb Andrea Gazzarini:

Hi,
looking at the stateful nature of SynonymGraphFilter/FilterFactory classes,
the answer should be 2 times (one time per type instance).
The SynonymMap, which internally holds the synonyms table, is a private
member of the filter factory and it is loaded each time the factory needs
to create a type.

Best,
Andrea

On 29/09/2019 23:49, Dominique Bejean wrote:

Hi,

My concern is about memory used by synonym filter, especially if synonyms
resources files are large.

If in my schema, there are two field types "TypeSyno1" and "TypeSyno2"
using synonym filter with the same synonyms files.
For each of these two field types there are two fields

1 2 3 >

1 - 100 of 246 matches

Mail list logo