Re: Solr 6: Use facet with Streaming Expressions- LeftOuterJoin
Thanks again ! I will try this and followup. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-6-Use-facet-with-Streaming-Expressions-LeftOuterJoin-tp4290526p4292341.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: DataImport-Solr6: Nested Entities
Well, do both parent and child entity have a field called 'id'
containing their corresponding unique ids? That would be the first
step.
Regards,
Alex.
Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/
On 19 August 2016 at 09:10, Peri Subrahmanya
wrote:
> Hi,
>
> I have a simple one-to-many relationship setup in the data-import.xml and
> when I try to index it using the dataImportHandler, Solr complains of “no
> unique id found”.
>
> managed-schema.xml
> id
> solrconfig,xml:
>
>
> id
>
>
>
>
>
> data-import.xml
>
> driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://x.x.x.x:3306/xx"
> user=“blah"
> password=“blah"/>
>
>
> query="select blah blah from catalog">
>
> query=“select blah blah from course where
> catalog_id=‘${catalog.catalog_id}'">
>
>
>
>
>
>
>
> Could someone please advise?
>
> Thanks
> -Peri
DataImport-Solr6: Nested Entities
Hi, I have a simple one-to-many relationship setup in the data-import.xml and when I try to index it using the dataImportHandler, Solr complains of “no unique id found”. managed-schema.xml id solrconfig,xml: id data-import.xml Could someone please advise? Thanks -Peri
Re: Error During Indexing - org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: early EOF
On 8/16/2016 8:53 PM, Jaspal Sawhney wrote: > We are running solr 4.6 in master-slave configuration where in our master is > used entirely for indexing. No search traffic comes to master ever. > Off late we have started to get the early EOF error on the solr Master which > results in a Broken Pipe error on the commerce application from where > Indexing was kicked off from. > 1. Since we are not using solrCloud – I want to understand the impact of > bypassing transaction log The transaction log is required for SolrCloud, but it is highly recommended for ANY Solr install. Here's how to remove issues with the tlog directory growing out of control: Configure autoCommit with openSearcher set to false, no maxDocs setting, and a maxTime of 6 (one minute). You'll commonly see 15 seconds as a recommended maxTime -- my opinion is that this is too frequent, but if you choose to use that, I doubt you'll have any issues. I think the newest versions of Solr do set up autoCommit like this with a 15 second maxTime. If you disable the transaction log and have some kind of crash during indexing, you may lose documents. When it is present, the transaction log will be replayed when the core starts. > 2. How does solr take documents which are sent to it to storage as in what > is the journey of a document from segment to tlog to storage. Assuming no cloud mode, when a document arrives for indexing, it is written to the tlog and sent to Lucene for processing. When the Lucene indexing buffer fills up, or a commit is issued, then the segment is flushed. Most of the time it will be flushed to disk, but if the segment is very small and a soft commit is used, it may be flushed to RAM instead -- this is a function of NRTCachingDirectoryFactory, which is the default. Cloud mode is slightly more complicated, but the behavior would be the same once the document arrives at the correct core(s) that will index it. The "early EOF" exception came from Jetty, not Solr. Based on how EOF is used by Jetty errors in other contexts, I think it means that the indexing client closed the connection before all the data was sent, which probably means that you have a low socket timeout on the client. The server likely paused while receiving the data, probably to handle the data it had already received ... and the pause was longer than the socket timeout, causing the client to close the connection. Another possibility is that the network is not working well, or that one of the operating systems or software libraries involved has TCP or HTTP bugs. I could be wrong about what the exception means, but the information I was able to quickly locate supports the idea. If I am right, then you will need to either reduce the amount of data that you send in a single update request, or increase the socket timeout that the indexing client is using on its connections. Erick's idea of your update request exceeding the maximum POST body size is something I hadn't thought of. The default for this limit is 2MB, and can be increased in solrconfig.xml. I suspect that this isn't the problem, but it's something to investigate. Thanks, Shawn
/select results different between 5.4 and 6.1
Hi all,
TL:DR -
Is it expected that the /select endpoint would produce different
scores/result order between versions 5.4 and 6.1?
(I'm aware that it's certainly possible I've done something different to
these environments, although at this point I can't see any difference in
configs etc... and I used a very simple search against /select to test this)
== Detail ==
I'm currently seeing different scoring and different result order when I
compare Solr results in the Admin console for a 5.4 and 6.1 environment.
I'm using the /select endpoint to try to avoid any difference in
configuration. To the best of my knowledge (and reading) I haven't ever
modified the xml for that endpoint.
As I was looking into it, I saw that the debug output looks quite different
in 6.1...
Any advice, including "You must have broken it yourself, that's impossible"
is much appreciated.
Here's debug from the "old" 5.4 SolrCloud environment. The id's are a pain
to read, but not only am I getting different scores, I'm getting different
docs (or docs in a clearly different order)
"debug": { "rawquerystring": "chiari", "querystring": "chiari", "parsedquery":
"text:chiari", "parsedquery_toString": "text:chiari", "explain": { "
d9644f86-5fe2-4a9f-8517-545e2cde0b64": "\n4.3581347 = weight(text:chiari in
26783) [ClassicSimilarity], result of:\n 4.3581347 = fieldWeight in 26783,
product of:\n 1.0 = tf(freq=1.0), with freq of:\n 1.0 = termFreq=1.0\n
6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=26783)\n", "1347f707-6fdd-4864-b9dd-6d3e7cc32bf5": "\n4.3581347
= weight(text:chiari in 26792) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 26792, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=26792)\n", "d01c32ad-e29d-4b65-9930-f8a6844a2613": "\n4.3581347
= weight(text:chiari in 27028) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27028, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27028)\n", "0c5a4be7-1162-4b1a-ab83-4b48a690fc3a": "\n4.3581347
= weight(text:chiari in 27029) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27029, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27029)\n", "e1cb441d-9d60-482d-956b-3fbc964a17c1": "\n4.3581347
= weight(text:chiari in 27042) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27042, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27042)\n", "f87951f1-e163-4f17-a628-904b9df0c609": "\n4.3581347
= weight(text:chiari in 27043) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27043, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27043)\n", "caaa7ca1-34cb-44a8-8dd9-12c909db8c2d": "\n4.3581347
= weight(text:chiari in 27044) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27044, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27044)\n", "ada7a87e-725a-4533-b72e-3817af4c7179": "\n4.3581347
= weight(text:chiari in 27055) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27055, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27055)\n", "ac6d47fd-9a59-47d6-8cfb-11b34c7ded54": "\n4.3581347
= weight(text:chiari in 27056) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 27056, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=27056)\n", "4aaa7697-b26a-4bea-ba4e-70d18ea649f0": "\n4.3581347
= weight(text:chiari in 62240) [ClassicSimilarity], result of:\n 4.3581347
= fieldWeight in 62240, product of:\n 1.0 = tf(freq=1.0), with freq of:\n
1.0 = termFreq=1.0\n 6.9730153 = idf(docFreq=281, maxDocs=110738)\n 0.625 =
fieldNorm(doc=62240)\n" }, "QParser": "LuceneQParser", "timing": { "time": 2,
"prepare": { "time": 0, "query": { "time": 0 },
... and here's the same from the Solr Cloud 6.0 environment
"debug":{ "rawquerystring":"chiari", "querystring":"chiari", "parsedquery":
"text:chiari", "parsedquery_toString":"text:chiari", "explain":{ "
85249c23-ef68-4276-9ef7-48c290033993":"\n9.735645 = weight(text:chiari in
106960) [], result of:\n 9.735645 = score(doc=106960,freq=50.0 =
termFreq=50.0\n), product of:\n 4.798444 = idf(docFreq=281,
docCount=34151)\n 2.0289173 = tfNorm, computed from:\n 50.0 =
termFreq=50.0\n 1.2 = parameter k1\n 0.75 = parameter b\n 941.3421 =
avgFieldLength\n 4096.0 = fieldLength\n", "
495b660d-8e8f-4b75-a523-106440468818":"\n9.655164 = w
Re: What does refCount denotes in solr admin
How are you indexing? It sounds like you may be sending one document at a time. Also, another common mistake is to send commits from the client, you should either set your autocommit intervals up in solrconfig.xml or, perhaps, send one (and only one) commit at the very end of the entire indexing job. Throughput is sensitive to the number of docs you send to Solr at once, see: https://lucidworks.com/blog/2015/10/05/really-batch-updates-solr-2/ Best, Erick On Thu, Aug 18, 2016 at 2:24 AM, kshitij tyagi wrote: > how to shrink solr container thread pool?? > > On Thu, Aug 18, 2016 at 2:53 PM, kshitij tyagi > wrote: > >> refcount 171 is seen when i reindex a number of documents simultaneously? >> what does this means? I am observing that my indexing speeds slows down >> when refcount increses. I am only indexing on this instance and no queries >> are running on it. >> >> Thanks for the information >> >> On Thu, Aug 18, 2016 at 2:20 PM, Mikhail Khludnev wrote: >> >>> When instance is idle you should see refCount=2 in solrAdmin. One count >>> goes from coreContainer holding a core instance until reload and two comes >>> from solrAdmin request which opens a core while it renders response. So, >>> until you don't request this stat the refCount is 1 that somehow remind >>> quantum mechanics. >>> Seen refcount 171 either mean 169 parallel request, which hardly make >>> sense >>> and I suggest to shrink the Solr container thread pool or/and tweak a >>> client app. But If it keeps growing and remains >2 even when instance is >>> idle, it means some plugin leak it - avoid to close/decref solr core. >>> Hope it helps. >>> >>> On Wed, Aug 17, 2016 at 5:27 PM, kshitij tyagi < >>> kshitij.shopcl...@gmail.com> >>> wrote: >>> >>> > any update?? >>> > >>> > On Wed, Aug 17, 2016 at 12:47 PM, kshitij tyagi < >>> > kshitij.shopcl...@gmail.com >>> > > wrote: >>> > >>> > > Hi, >>> > > >>> > > I need to understand what is refcount in stats section of solr admin. >>> > > >>> > > I am seeing refcount: 2 on my solr cores and on one of the core i am >>> > > seeing refcount:171. >>> > > >>> > > The core with refcount with higher number is having very slow >>> indexing >>> > > speed? >>> > > >>> > > >>> > > >>> > >>> >>> >>> >>> -- >>> Sincerely yours >>> Mikhail Khludnev >>> >> >>
Solr 6.1.0, zookeeper 3.4.8, Solrj and SolrCloud
Hi SolrTeam,
I see session exipre and my solr index fails.
please help me here, my infra details are shared below
I have total 3 compute
nodes[pcam-stg-app-02,pcam-stg-app-03,pcam-stg-app-04]
1) 3 nodes are running with zoo1, zoo2, zoo3 instances
/apps/scm-core/zookeeper/zkData/zkData1/myid value 1
/apps/scm-core/zookeeper/zkData/zkData2/myid value 2
/apps/scm-core/zookeeper/zkData/zkData3/myid value 3
zoo1.cfg my setup
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/apps/scm-core/zookeeper/zkData/zkData1
clientPort=2181
server.1=pcam-stg-app-01:2888:3888
server.2=pcam-stg-app-02:2888:3888
server.3=pcam-stg-app-03:2888:3888
server.4=pcam-stg-app-04:2888:3888
dataLogDir=/apps/scm-core/zookeeper/zkLogData/zkLogData1
# Default 64M, changed to 128M, represented in KiloBytes
preAllocSize=131072
# Default : 10
snapCount=100
globalOutstandingLimit=1000
maxClientCnxns=100
autopurge.snapRetainCount=3
autopurge.purgeInterval=23
minSessionTimeout=4
maxSessionTimeout=30
[zk: pcam-stg-app-02:2181(CONNECTED) 0] ls /
[zookeeper, solr]
[zk: pcam-stg-app-02:2181(CONNECTED) 1] ls /solr
[configs, overseer, aliases.json, live_nodes, collections, overseer_elect,
security.json, clusterstate.json]
2) 2 nodes are running solrcloud
pcam-stg-app-03: solr port 8983, solr port 8984
pcam-stg-app-04: solr port 8983, solr port 8984
Config upload to zookeeper
server/scripts/cloud-scripts/zkcli.sh -zkhost
pcam-stg-app-02:2181,pcam-stg-app-03:2181,pcam-stg-app-04:2181/solr \
-cmd upconfig -confname scdata -confdir
/apps/scm-core/solr/solr-6.1.0/server/solr/configsets/data_driven_schema_configs/conf
Collection creation url:
http://pcam-stg-app-03:8983/solr/admin/collections?action=CREATE&name=scdata_test&numShards=2&replicationFactor=2&maxShardsPerNode=2&createNodeSet=pcam-stg-app-03:8983_solr,pcam-stg-app-03:8984_solr,pcam-stg-app-04:8983_solr,pcam-stg-app-04:8984_solr&collection.configName=scdata
solrj client
String zkHosts =
"pcam-stg-app-02:2181,pcam-stg-app-03:2181,pcam-stg-app-04:2181/solr";
CloudSolrClient solrClient = new
CloudSolrClient.Builder().withZkHost(zkHosts).build();
solrClient.setDefaultCollection("scdata_test");
solrClient.setParallelUpdates(true);
List cpnSpendSavingsList = new ArrayList<>();
i have done data setter to cpnSpendSavingsList
solrClient.addBeans(cpnSpendSavingsList);
solrClient.commit();
SessionExpire Error for the collections
Why this SessionExpire error comes when i start bulk insert/update to solr
org.apache.solr.common.SolrException: Could not load collection from ZK:
scdata_test
at
org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1047)
at
org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:610)
at
org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:211)
at
org.apache.solr.common.cloud.ClusterState.hasCollection(ClusterState.java:113)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1239)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:961)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
at
org.apache.solr.client.solrj.SolrClient.addBeans(SolrClient.java:357)
at
org.apache.solr.client.solrj.SolrClient.addBeans(SolrClient.java:329)
at
com.cisco.pcam.spark.stream.HiveDataProcessStream.main(HiveDataProcessStream.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)
at
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
at
org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException:
KeeperErrorCode = Session expired for /collections/scdata_test/state.json
Re: Solr 6: Use facet with Streaming Expressions- LeftOuterJoin
Ah, the documentation is not clear on this. We need to fix that. The rollup() function supports multi-dimension aggregations like the facet expression. You just provide a comma delimited list of fields in the over parameter: over="a,b,c" You will need to sort the underlying stream by those fields well. sort="a asc, b asc, c asc" Joel Bernstein http://joelsolr.blogspot.com/ On Thu, Aug 18, 2016 at 5:42 AM, vrindavda wrote: > I am not able to get count(*) for more than one field > > > > -- > View this message in context: http://lucene.472066.n3. > nabble.com/Solr-6-Use-facet-with-Streaming-Expressions-LeftOuterJoin- > tp4290526p4292208.html > Sent from the Solr - User mailing list archive at Nabble.com. >
Antw: Re: Solr 6.1 :: Logging -> WARN FieldTypePluginLoader
... thank you! Great answer. This saved me a lot of time. Rainer Gnan Bayerische Staatsbibliothek BibliotheksVerbund Bayern Verbundnahe Dienste 80539 München Tel.: +49(0)89/28638-2445 Fax: +49(0)89/28638-2665 E-Mail: rainer.g...@bsb-muenchen.de >>> Shawn Heisey 18.08.2016 15:11 >>> On 8/18/2016 3:42 AM, Rainer Gnan wrote: > after upgrading from "Solr 5.3" to "Solr 6.1" I get the following > logging message: -> WARN true FieldTypePluginLoader TokenFilterFactory > is using deprecated 5.2.1 emulation. You should at some point declare > and reindex to at least 6.0, because 5.x emulation is > deprecated and will be removed in 7.0 The "luceneMatchVersion" setting in your solrconfig.xml file is 5.2.1. It might also appear in schema.xml, but solrconfig.xml is more likely. The message is saying "right now, the config settings you're using will be honored, but they won't be in the future." This message is a warning and will not cause Solr to blow up, even in 7.0. The first thing you'll need to do is change luceneMatchVersion to match your Solr version -- 6.1.0 -- and reindex. https://wiki.apache.org/solr/HowToReindex Note that this may change how your analyzer chains defined in schema.xml work. The change may be obvious, it may be subtle, and in some cases it might actually make no change at all. I don't have a list of changes that are made by different luceneMatchVersion settings. One thing that luceneMatchVersion will *not* change is index format. I mention this because this is a common misconception. The index format will be decided by the version of Solr (Lucene) that you're running, not luceneMatchVersion. Thanks, Shawn
Re: Solr date range returns wrong number of results
On 8/18/2016 1:43 AM, Tali Finelt wrote: > I am using Solr 6 in SolrCloud mode and trying to filter results according > to a date range. > > The date field is defined as following in the managed_schema file: > required="false" multiValued="false" /> > > When I run the following query: > > https://:8983/solr//select?q=date:[2016-08-02T04:43:38Z > TO 2016-08-02T05:43:38Z]&rows=10 > For some reason, I only get 7 documents back and not 10 as I asked (by > parameter "rows=10"). > I know there are more then 7 documents according to the "numFound" > parameter (=325313) My best guess, which could be wrong, is that you have documents with the same uniqueKey in more than one shard of your collection. With SolrCloud, this usually happens when the router is implicit, or when routing is overridden. The implicit router is correctly named, but the meaning is often misunderstood. The best interpretation of its meaning is "manual routing." When this problem happens you may find that numFound sometimes changes when running the same query more than once, even when the index hasn't changed. What router are you using on your collection, and are you influencing the routing when you index? > This problem persists on different date ranges I query on. > This does not happen when I filter on other fields in my schema. > > What am I doing wrong? > How can I get 10 results back? If I'm right about what's happening, then the fix would be to eliminate the duplicate documents. Side issue: Is that a literal paste of the full response, or were there contents inside the tags? The XML that was pasted had plus signs next to the tags, which might have come from those tags being collapsed. Thanks, Shawn
Re: Solr 6.1 :: Logging -> WARN FieldTypePluginLoader
On 8/18/2016 3:42 AM, Rainer Gnan wrote: > after upgrading from "Solr 5.3" to "Solr 6.1" I get the following > logging message: -> WARN true FieldTypePluginLoader TokenFilterFactory > is using deprecated 5.2.1 emulation. You should at some point declare > and reindex to at least 6.0, because 5.x emulation is > deprecated and will be removed in 7.0 The "luceneMatchVersion" setting in your solrconfig.xml file is 5.2.1. It might also appear in schema.xml, but solrconfig.xml is more likely. The message is saying "right now, the config settings you're using will be honored, but they won't be in the future." This message is a warning and will not cause Solr to blow up, even in 7.0. The first thing you'll need to do is change luceneMatchVersion to match your Solr version -- 6.1.0 -- and reindex. https://wiki.apache.org/solr/HowToReindex Note that this may change how your analyzer chains defined in schema.xml work. The change may be obvious, it may be subtle, and in some cases it might actually make no change at all. I don't have a list of changes that are made by different luceneMatchVersion settings. One thing that luceneMatchVersion will *not* change is index format. I mention this because this is a common misconception. The index format will be decided by the version of Solr (Lucene) that you're running, not luceneMatchVersion. Thanks, Shawn
Re: Solr 6: Use facet with Streaming Expressions- LeftOuterJoin
I am not able to get count(*) for more than one field -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-6-Use-facet-with-Streaming-Expressions-LeftOuterJoin-tp4290526p4292208.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr 6.1 :: Logging -> WARN FieldTypePluginLoader
Hi, after upgrading from "Solr 5.3" to "Solr 6.1" I get the following logging message: -> WARN true FieldTypePluginLoader TokenFilterFactory is using deprecated 5.2.1 emulation. You should at some point declare and reindex to at least 6.0, because 5.x emulation is deprecated and will be removed in 7.0 I could not found relevant information in the web or in other resources in order to implement or play out this tip: You should at some point declare and reindex to at least 6.0, Neither solrconfig.xml nor schema.xml contains the mentioned Class "TokenFilterFactory ". schema.xml contains only "solr.RemoveDuplicatesToTokenFilterFactory" ... Could anybody give me a hint? I appreciate any help. Rainer Rainer Gnan Bayerische Staatsbibliothek BibliotheksVerbund Bayern Verbundnahe Dienste 80539 München Tel.: +49(0)89/28638-2445 Fax: +49(0)89/28638-2665 E-Mail: rainer.g...@bsb-muenchen.de
Re: What does refCount denotes in solr admin
how to shrink solr container thread pool?? On Thu, Aug 18, 2016 at 2:53 PM, kshitij tyagi wrote: > refcount 171 is seen when i reindex a number of documents simultaneously? > what does this means? I am observing that my indexing speeds slows down > when refcount increses. I am only indexing on this instance and no queries > are running on it. > > Thanks for the information > > On Thu, Aug 18, 2016 at 2:20 PM, Mikhail Khludnev wrote: > >> When instance is idle you should see refCount=2 in solrAdmin. One count >> goes from coreContainer holding a core instance until reload and two comes >> from solrAdmin request which opens a core while it renders response. So, >> until you don't request this stat the refCount is 1 that somehow remind >> quantum mechanics. >> Seen refcount 171 either mean 169 parallel request, which hardly make >> sense >> and I suggest to shrink the Solr container thread pool or/and tweak a >> client app. But If it keeps growing and remains >2 even when instance is >> idle, it means some plugin leak it - avoid to close/decref solr core. >> Hope it helps. >> >> On Wed, Aug 17, 2016 at 5:27 PM, kshitij tyagi < >> kshitij.shopcl...@gmail.com> >> wrote: >> >> > any update?? >> > >> > On Wed, Aug 17, 2016 at 12:47 PM, kshitij tyagi < >> > kshitij.shopcl...@gmail.com >> > > wrote: >> > >> > > Hi, >> > > >> > > I need to understand what is refcount in stats section of solr admin. >> > > >> > > I am seeing refcount: 2 on my solr cores and on one of the core i am >> > > seeing refcount:171. >> > > >> > > The core with refcount with higher number is having very slow >> indexing >> > > speed? >> > > >> > > >> > > >> > >> >> >> >> -- >> Sincerely yours >> Mikhail Khludnev >> > >
Re: What does refCount denotes in solr admin
refcount 171 is seen when i reindex a number of documents simultaneously? what does this means? I am observing that my indexing speeds slows down when refcount increses. I am only indexing on this instance and no queries are running on it. Thanks for the information On Thu, Aug 18, 2016 at 2:20 PM, Mikhail Khludnev wrote: > When instance is idle you should see refCount=2 in solrAdmin. One count > goes from coreContainer holding a core instance until reload and two comes > from solrAdmin request which opens a core while it renders response. So, > until you don't request this stat the refCount is 1 that somehow remind > quantum mechanics. > Seen refcount 171 either mean 169 parallel request, which hardly make sense > and I suggest to shrink the Solr container thread pool or/and tweak a > client app. But If it keeps growing and remains >2 even when instance is > idle, it means some plugin leak it - avoid to close/decref solr core. > Hope it helps. > > On Wed, Aug 17, 2016 at 5:27 PM, kshitij tyagi < > kshitij.shopcl...@gmail.com> > wrote: > > > any update?? > > > > On Wed, Aug 17, 2016 at 12:47 PM, kshitij tyagi < > > kshitij.shopcl...@gmail.com > > > wrote: > > > > > Hi, > > > > > > I need to understand what is refcount in stats section of solr admin. > > > > > > I am seeing refcount: 2 on my solr cores and on one of the core i am > > > seeing refcount:171. > > > > > > The core with refcount with higher number is having very slow > indexing > > > speed? > > > > > > > > > > > > > > > -- > Sincerely yours > Mikhail Khludnev >
Re: What does refCount denotes in solr admin
When instance is idle you should see refCount=2 in solrAdmin. One count goes from coreContainer holding a core instance until reload and two comes from solrAdmin request which opens a core while it renders response. So, until you don't request this stat the refCount is 1 that somehow remind quantum mechanics. Seen refcount 171 either mean 169 parallel request, which hardly make sense and I suggest to shrink the Solr container thread pool or/and tweak a client app. But If it keeps growing and remains >2 even when instance is idle, it means some plugin leak it - avoid to close/decref solr core. Hope it helps. On Wed, Aug 17, 2016 at 5:27 PM, kshitij tyagi wrote: > any update?? > > On Wed, Aug 17, 2016 at 12:47 PM, kshitij tyagi < > kshitij.shopcl...@gmail.com > > wrote: > > > Hi, > > > > I need to understand what is refcount in stats section of solr admin. > > > > I am seeing refcount: 2 on my solr cores and on one of the core i am > > seeing refcount:171. > > > > The core with refcount with higher number is having very slow indexing > > speed? > > > > > > > -- Sincerely yours Mikhail Khludnev
Re: Tagging and excluding Filters with BlockJoin Queries and BlockJoin Faceting
The last comment at https://issues.apache.org/jira/browse/SOLR-8998 shows
the current verbose json.facet syntax which provides aggregated facet
counts already. It's a little bit slower that child.facet.field.
Nevertheless, you can take this sample and add exclusion instructions into.
It should work. Let me know how does it, please.
On Wed, Aug 17, 2016 at 5:35 PM, Stefan Moises wrote:
> Hi Mikhail,
>
> thanks for the info ... what is the advantage of using the JSON FACET API
> compared to the standard BlockJoinQuery features?
>
> Is there already anybody working on the tagging/exclusion feature or is
> there any timeframe for it? There wasn't any discussion yet in SOLR-8998
> about exclusions, was there?
>
> Thank you very much,
>
> best,
>
> Stefan
>
>
> Am 17.08.16 um 15:26 schrieb Mikhail Khludnev:
>
> Stefan,
>> child.facet.field never intend to support exclusions. My preference is to
>> implement it under json.facet that's discussed under
>> https://issues.apache.org/jira/browse/SOLR-8998.
>>
>> On Wed, Aug 17, 2016 at 3:52 PM, Stefan Moises
>> wrote:
>>
>> Hey girls and guys,
>>>
>>> for a long time we have been using our own BlockJoin Implementation,
>>> because for our Shop Systems a lot of requirements that we had were not
>>> implemented in solr.
>>>
>>> As we now had a deeper look into how far the standard has come, we saw
>>> that BlockJoin and faceting on children is now part of the standard,
>>> which
>>> is pretty cool.
>>> When I tried to refactor our external code to use that now, I stumbled
>>> upon one non-working feature with BlockJoins that still keeps us from
>>> using
>>> it:
>>>
>>> It seems that tagging and excluding Filters with BlockJoin Faceting
>>> simply
>>> does not work yet.
>>>
>>> Simple query:
>>>
>>> &qt=products
>>> &q={!parent which='isparent:true'}shirt AND isparent:false
>>> &facet=true
>>> &fq={!parent which='isparent:true'}{!tag=myTag}color:grey
>>> &child.facet.field={!ex=myTag}color
>>>
>>>
>>> Gives us:
>>> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException:
>>> undefined field: "{!ex=myTag}color"
>>> at org.apache.solr.schema.IndexSchema.getField(IndexSchema.
>>> java:1231)
>>>
>>>
>>> Does somebody have an idea?
>>>
>>>
>>> Best,
>>> Stefan
>>>
>>> --
>>> --
>>>
>>> Stefan Moises
>>> Manager Research & Development
>>> shoptimax GmbH
>>> Ulmenstraße 52 H
>>> 90443 Nürnberg
>>> Tel.: 0911/25566-0
>>> Fax: 0911/25566-29
>>> moi...@shoptimax.de
>>> http://www.shoptimax.de
>>>
>>> Geschäftsführung: Friedrich Schreieck
>>> Ust.-IdNr.: DE 814340642
>>> Amtsgericht Nürnberg HRB 21703
>>>
>>>
>>>
>>>
>>
> --
> --
>
> Stefan Moises
> Manager Research & Development
> shoptimax GmbH
> Ulmenstraße 52 H
> 90443 Nürnberg
> Tel.: 0911/25566-0
> Fax: 0911/25566-29
> moi...@shoptimax.de
> http://www.shoptimax.de
>
> Geschäftsführung: Friedrich Schreieck
> Ust.-IdNr.: DE 814340642
> Amtsgericht Nürnberg HRB 21703
>
>
>
--
Sincerely yours
Mikhail Khludnev
Solr date range returns wrong number of results
Hi All, I am using Solr 6 in SolrCloud mode and trying to filter results according to a date range. The date field is defined as following in the managed_schema file: When I run the following query: https://:8983/solr//select?q=date:[2016-08-02T04:43:38Z TO 2016-08-02T05:43:38Z]&rows=10 I get the following results: true 0 20 date:[2016-08-02T04:43:38Z TO 2016-08-02T05:43:38Z] 10 + + + + + + + For some reason, I only get 7 documents back and not 10 as I asked (by parameter "rows=10"). I know there are more then 7 documents according to the "numFound" parameter (=325313) This problem persists on different date ranges I query on. This does not happen when I filter on other fields in my schema. What am I doing wrong? How can I get 10 results back? Thanks, Tali
