Re: what does UnInvertedField; UnInverted multi-valued field means and how to fix it
Hay, thanks for the reply.
So after a full day spent only on trying to figure this out, I have found
the cause (spellcheck component)... but not the solution.
Se my other post with the subject *spellcheck causing Core Reload to hang*.
I have explained it there.
Thanks a lot.
On Sun, Sep 15, 2013 at 2:35 AM, Erick Erickson erickerick...@gmail.comwrote:
This is totally weird. Can you give us the exact
command you are using?
Best
Erick
On Fri, Sep 13, 2013 at 8:15 AM, Raheel Hasan raheelhasan@gmail.com
wrote:
Hi guyz,
I have an issue here in between Solr Core and Data Indexing:
When I build some index from fresh setup, everything is fine: all queries
and additional/update indexing, everything runs is fine. But when I
reload
the Core, the solr stops from that point onward forever.
All i get is this line as the last line of the solr log after the issue
as
occurred:
UnInvertedField; UnInverted multi-valued field
{field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
Furthermore, the only way to get things working again, would be to delete
the data folder inside solr/{myCore}/...
So can anyone help me beat this issue and get things working again? I
cant
afford this issue when the system is LIVE..
Thanks a lot.
--
Regards,
Raheel Hasan
--
Regards,
Raheel Hasan
Re: what does UnInvertedField; UnInverted multi-valued field means and how to fix it
This is totally weird. Can you give us the exact
command you are using?
Best
Erick
On Fri, Sep 13, 2013 at 8:15 AM, Raheel Hasan raheelhasan@gmail.comwrote:
Hi guyz,
I have an issue here in between Solr Core and Data Indexing:
When I build some index from fresh setup, everything is fine: all queries
and additional/update indexing, everything runs is fine. But when I reload
the Core, the solr stops from that point onward forever.
All i get is this line as the last line of the solr log after the issue as
occurred:
UnInvertedField; UnInverted multi-valued field
{field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
Furthermore, the only way to get things working again, would be to delete
the data folder inside solr/{myCore}/...
So can anyone help me beat this issue and get things working again? I cant
afford this issue when the system is LIVE..
Thanks a lot.
--
Regards,
Raheel Hasan
what does UnInvertedField; UnInverted multi-valued field means and how to fix it
Hi guyz,
I have an issue here in between Solr Core and Data Indexing:
When I build some index from fresh setup, everything is fine: all queries
and additional/update indexing, everything runs is fine. But when I reload
the Core, the solr stops from that point onward forever.
All i get is this line as the last line of the solr log after the issue as
occurred:
UnInvertedField; UnInverted multi-valued field
{field=prod_cited_id,memSize=4880,tindexSize=40,time=4,phase1=4,nTerms=35,bigTerms=4,termInstances=36,uses=0}
Furthermore, the only way to get things working again, would be to delete
the data folder inside solr/{myCore}/...
So can anyone help me beat this issue and get things working again? I cant
afford this issue when the system is LIVE..
Thanks a lot.
--
Regards,
Raheel Hasan
Re: UnInverted multi-valued field
Hello,
well ... we have 5 multi-valued facet fields, so you had to wait
sometimes up to one minute.
The old searcher blocks during this time.
@Toke Eskildsen: the example I posted was a very small update, usually
there are more terms.
We are using Solr 3.6. I don't know if it will be faster with 4.x.
These are the configurations of our cache:
filterCache
class=solr.FastLRUCache
size=30
initialSize=30
autowarmCount=5/
queryResultCache
class=solr.LRUCache
size=10
initialSize=10
autowarmCount=5/
documentCache
class=solr.LRUCache
size=5
initialSize=5
autowarmCount=1/
We have 5 million document in our index.
@Roman: Do you think our autowarmCound should be larger?
Greetings
Jochen
Roman Chyla schrieb:
On Wed, Jun 19, 2013 at 5:30 AM, Jochen Lienhard
lienh...@ub.uni-freiburg.de wrote:
Hi @all.
We have the problem that after an update the index takes to much time for
'warm up'.
We have some multivalued facet-fields and during the startup solr creates
the messages:
INFO: UnInverted multi-valued field {field=mt_facet,memSize=**
18753256,tindexSize=54,time=**170,phase1=156,nTerms=17,**
bigTerms=3,termInstances=**903276,uses=0}
In the solconfig we use the facet.method 'fc'.
We know, that the start-up with the method 'enum' is faster, but then the
searches are very slow.
How do you handle this problem?
Or have you any idea for optimizing the warm up?
Or what do you do after an update?
You probably know, but just in case... you may use autowarming; the
searcher will populate the cache and only after the warmup queries
finished, will it be exposed to the world. The old searcher continues to
handle requests in the meantime.
roman
Greetings
Jochen
--
Dr. rer. nat. Jochen Lienhard
Dezernat EDV
Albert-Ludwigs-Universität Freiburg
Universitätsbibliothek
Rempartstr. 10-16 | Postfach 1629
79098 Freiburg | 79016 Freiburg
Telefon: +49 761 203-3908
E-Mail: lienh...@ub.uni-freiburg.de
Internet: www.ub.uni-freiburg.de
--
Dr. rer. nat. Jochen Lienhard
Dezernat EDV
Albert-Ludwigs-Universität Freiburg
Universitätsbibliothek
Rempartstr. 10-16 | Postfach 1629
79098 Freiburg | 79016 Freiburg
Telefon: +49 761 203-3908
E-Mail: lienh...@ub.uni-freiburg.de
Internet: www.ub.uni-freiburg.de
Re: UnInverted multi-valued field
Hello, Am 20.06.2013 09:34, schrieb Jochen Lienhard: Hello, well ... we have 5 multi-valued facet fields, so you had to wait sometimes up to one minute. The old searcher blocks during this time. May be related to an already fixed SOLR-4589 issue? Generally there is no blocking by the old searcher. It just feels like blocking because the system is busy with your tons of autowarming so that the old searcher has no chance to answer queries. @Toke Eskildsen: the example I posted was a very small update, usually there are more terms. We are using Solr 3.6. I don't know if it will be faster with 4.x. DocValues are introduced to SOLR with version 4.2 It is always good to use a more recent version because of improvements, bug fixes, new features,... These are the configurations of our cache: filterCache class=solr.FastLRUCache size=30 initialSize=30 autowarmCount=5/ queryResultCache class=solr.LRUCache size=10 initialSize=10 autowarmCount=5/ documentCache class=solr.LRUCache size=5 initialSize=5 autowarmCount=1/ We have 5 million document in our index. So how have you calculated these values? Looks like they were just set by chance. autowarmCount depends on what your system is serving and how it is configured. Like my system with 46 million docs I have autowarmCount=0 for all and do a static warming. Why static warming? Because I have a static system. As an example you have queryResultCache set to 10 which means IF you calculate with 100 qps (which is a lot) it will cache the last 1000 seconds (if all queries are unique) which is 16.6 minutes. Is that what you want and what your system should serve? And with the halfe of it (5) a new searcher should be warmed? Regards, Bernd -- * Bernd FehlingBielefeld University Library Dipl.-Inform. (FH)LibTec - Library Technology Universitätsstr. 25 and Knowledge Management 33615 Bielefeld Tel. +49 521 106-4060 bernd.fehling(at)uni-bielefeld.de BASE - Bielefeld Academic Search Engine - www.base-search.net *
UnInverted multi-valued field
Hi @all.
We have the problem that after an update the index takes to much time
for 'warm up'.
We have some multivalued facet-fields and during the startup solr
creates the messages:
INFO: UnInverted multi-valued field
{field=mt_facet,memSize=18753256,tindexSize=54,time=170,phase1=156,nTerms=17,bigTerms=3,termInstances=903276,uses=0}
In the solconfig we use the facet.method 'fc'.
We know, that the start-up with the method 'enum' is faster, but then
the searches are very slow.
How do you handle this problem?
Or have you any idea for optimizing the warm up?
Or what do you do after an update?
Greetings
Jochen
--
Dr. rer. nat. Jochen Lienhard
Dezernat EDV
Albert-Ludwigs-Universität Freiburg
Universitätsbibliothek
Rempartstr. 10-16 | Postfach 1629
79098 Freiburg | 79016 Freiburg
Telefon: +49 761 203-3908
E-Mail: lienh...@ub.uni-freiburg.de
Internet: www.ub.uni-freiburg.de
Re: UnInverted multi-valued field
Take a look at using DocValues for faceted fields.
-- Jack Krupansky
-Original Message-
From: Jochen Lienhard
Sent: Wednesday, June 19, 2013 5:30 AM
To: solr-user@lucene.apache.org
Subject: UnInverted multi-valued field
Hi @all.
We have the problem that after an update the index takes to much time
for 'warm up'.
We have some multivalued facet-fields and during the startup solr
creates the messages:
INFO: UnInverted multi-valued field
{field=mt_facet,memSize=18753256,tindexSize=54,time=170,phase1=156,nTerms=17,bigTerms=3,termInstances=903276,uses=0}
In the solconfig we use the facet.method 'fc'.
We know, that the start-up with the method 'enum' is faster, but then
the searches are very slow.
How do you handle this problem?
Or have you any idea for optimizing the warm up?
Or what do you do after an update?
Greetings
Jochen
--
Dr. rer. nat. Jochen Lienhard
Dezernat EDV
Albert-Ludwigs-Universität Freiburg
Universitätsbibliothek
Rempartstr. 10-16 | Postfach 1629
79098 Freiburg | 79016 Freiburg
Telefon: +49 761 203-3908
E-Mail: lienh...@ub.uni-freiburg.de
Internet: www.ub.uni-freiburg.de
Re: UnInverted multi-valued field
On Wed, 2013-06-19 at 11:30 +0200, Jochen Lienhard wrote:
INFO: UnInverted multi-valued field
{field=mt_facet,memSize=18753256,tindexSize=54,time=170,phase1=156,nTerms=17,bigTerms=3,termInstances=903276,uses=0}
170ms does not sound like much to me. What are you hoping for?
We know, that the start-up with the method 'enum' is faster, but then
the searches are very slow.
That is a bit strange. With only 17 terms, enum should be quite fast.
How much do the two methods differ in speed?
- Toke Eskildsen
Re: UnInverted multi-valued field
On Wed, Jun 19, 2013 at 5:30 AM, Jochen Lienhard
lienh...@ub.uni-freiburg.de wrote:
Hi @all.
We have the problem that after an update the index takes to much time for
'warm up'.
We have some multivalued facet-fields and during the startup solr creates
the messages:
INFO: UnInverted multi-valued field {field=mt_facet,memSize=**
18753256,tindexSize=54,time=**170,phase1=156,nTerms=17,**
bigTerms=3,termInstances=**903276,uses=0}
In the solconfig we use the facet.method 'fc'.
We know, that the start-up with the method 'enum' is faster, but then the
searches are very slow.
How do you handle this problem?
Or have you any idea for optimizing the warm up?
Or what do you do after an update?
You probably know, but just in case... you may use autowarming; the
searcher will populate the cache and only after the warmup queries
finished, will it be exposed to the world. The old searcher continues to
handle requests in the meantime.
roman
Greetings
Jochen
--
Dr. rer. nat. Jochen Lienhard
Dezernat EDV
Albert-Ludwigs-Universität Freiburg
Universitätsbibliothek
Rempartstr. 10-16 | Postfach 1629
79098 Freiburg | 79016 Freiburg
Telefon: +49 761 203-3908
E-Mail: lienh...@ub.uni-freiburg.de
Internet: www.ub.uni-freiburg.de
solr.TrieFloatField with multiValued=false treated as `UnInverted multi-valued field`
Hi!
I have problem with coding own SearchComponent. My schema.xml is:
...
fieldType name=decimal class=solr.TrieFloatField
precisionStep=2 omitNorms=true positionIncrementGap=0 /
...
field name=price_min type=decimal indexed=true
stored=true multiValued=false /
...
When I use this value in my code leaves this in log:
Jul 28, 2011 4:29:04 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field
{field=price_min,memSize=13758712,tindexSize=28852,time=2407,phase1=2398,nTerms=184366,bigTerms=0,termInstances=3248049,uses=0}
So it suggest that `price_min` is mutliValue but it isn't and I'm confused.
In code those values are also false:
SchemaField sf = searcher.getSchema().getField(field);
FieldType ft = sf.getType();
sf.multiValued() || ft.multiValuedFieldCache() // is false
I don't also understand why for this value is used UnInvertedField. Could
anybody explain it to me? I'm confused when I try:
FieldCache.StringIndex si =
FieldCache.DEFAULT.getStringIndex(searcher.getReader(), fieldName);
String termText = si.lookup[si.order[docID]];
And here `termText` is everytime equal to '~' and si.order[docID] is at the
end of their array.
It would be great to see from someone any helpful idea.
--
Rafał RaVbaker Piekarski.
facet search and UnInverted multi-valued field?
Dear list,
we use solr 3.1.0.
my logs have the following entry:
May 3, 2011 2:01:39 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field
{field=f_dcperson,memSize=1966237,tindexSize=35730,time=849,phase1=782,nTerms=12,bigTerms=0,termInstances=368008,uses=0}
The schema.xml has the field:
field name=f_dcperson type=string indexed=true stored=true
multiValued=true /
The query was:
May 3, 2011 2:01:40 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={facet=truefl=scorefacet.mincount=1facet.sort=start=0event=firstSearcherq=text:antigone^200facet.prefix=facet.limit=100facet.field=f_dcpersonfacet.field=f_dcsubjectfacet.field=f_dcyearfacet.field=f_dccollectionfacet.field=f_dctypenormfacet.field=f_dccontenttyperows=10}
hits=1 status=0 QTime=1816
At first the log entry is an info, but what does it tell me?
Am I doing something wrong or can something be done better?
Regards,
Bernd
Re: facet search and UnInverted multi-valued field?
UnInvertedField is similar to Lucene's FieldCache, except, while the
FieldCache cannot work with multivalued fields, UnInvertedField is designed
for that very purpose. So since your f_dcperson field is multivalued, by
default you use UnInvertedField. You're not doing anything wrong, that's
default and normal behavior.
-Jay
http://lucidimagination.com
On Tue, May 3, 2011 at 7:03 AM, Bernd Fehling
bernd.fehl...@uni-bielefeld.de wrote:
Dear list,
we use solr 3.1.0.
my logs have the following entry:
May 3, 2011 2:01:39 PM org.apache.solr.request.UnInvertedField uninvert
INFO: UnInverted multi-valued field
{field=f_dcperson,memSize=1966237,tindexSize=35730,time=849,phase1=782,nTerms=12,bigTerms=0,termInstances=368008,uses=0}
The schema.xml has the field:
field name=f_dcperson type=string indexed=true stored=true
multiValued=true /
The query was:
May 3, 2011 2:01:40 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=null
params={facet=truefl=scorefacet.mincount=1facet.sort=start=0event=firstSearcherq=text:antigone^200facet.prefix=facet.limit=100facet.field=f_dcpersonfacet.field=f_dcsubjectfacet.field=f_dcyearfacet.field=f_dccollectionfacet.field=f_dctypenormfacet.field=f_dccontenttyperows=10}
hits=1 status=0 QTime=1816
At first the log entry is an info, but what does it tell me?
Am I doing something wrong or can something be done better?
Regards,
Bernd
