Re:

2020-05-12 Thread Bernd Fehling
Dear list and mailer admins,

it looks like the mailer of this list needs some care .
Can someone please set this "ART GALLERY" on a black list?

Thank you,
Bernd


Am 13.05.20 um 08:47 schrieb ART GALLERY:
> check out the videos on this website TROO.TUBE don't be such a
> sheep/zombie/loser/NPC. Much love!
> https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219
> 
> On Tue, May 12, 2020 at 9:16 AM Nikolai Efseaff  wrote:
>>
>>
>>
>>
>> Any tax advice in this e-mail should be considered in the context of the tax 
>> services we are providing to you. Preliminary tax advice should not be 
>> relied upon and may be insufficient for penalty protection.
>> 
>> The information contained in this message may be privileged and confidential 
>> and protected from disclosure. If the reader of this message is not the 
>> intended recipient, or an employee or agent responsible for delivering this 
>> message to the intended recipient, you are hereby notified that any 
>> dissemination, distribution or copying of this communication is strictly 
>> prohibited. If you have received this communication in error, please notify 
>> us immediately by replying to the message and deleting it from your computer.
>>
>> Notice required by law: This e-mail may constitute an advertisement or 
>> solicitation under U.S. law, if its primary purpose is to advertise or 
>> promote a commercial product or service. You may choose not to receive 
>> advertising and promotional messages from Ernst & Young LLP (except for EY 
>> Client Portal and the ey.com website, which track e-mail preferences through 
>> a separate process) at this e-mail address by forwarding this message to 
>> no-more-m...@ey.com. If you do so, the sender of this message will be 
>> notified promptly. Our principal postal address is 5 Times Square, New York, 
>> NY 10036. Thank you. Ernst & Young LLP


Re: 8.5.1 LogReplayer extremely slow

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 6:23 AM Markus Jelsma
 wrote:
>
> I found the bastard, it was a freaky document that skrewed Solr over, 
> indexing kept failing, passing documents between replica's times out, 
> documents get reindexed and so the document (and others) end up in the 
> transaction log (many times) and are eligible for reindexing. Reindexing and 
> replaying of the transaction log both fail on that specific document. 
> Recovery was also not possible due to time outs.
>
> Although the original document [1] is a mess, Solr should have no 
> difficulties ingesting it [2]. Any ideas what is going on? Ticket, if so, 
> about what exactly? For the record, this is PreAnalyzed.
>
> Many thanks,
> Markus
>
> [1] https://pastebin.com/1NqBdYCM
> [2] https://www.openindex.io/export/do_not_index.xml
>
> -Original message-
> > From:Markus Jelsma 
> > Sent: Monday 11th May 2020 18:43
> > To: solr-user 
> > Subject: 8.5.1 LogReplayer extremely slow
> >
> > Hello,
> >
> > Our main Solr text search collection broke down last night (search was 
> > still working fine), every indexing action timed out with the Solr master 
> > spending most of its time in Java regex. One shard has only one replica 
> > left for queries and it stays like that. I have copied both shard's leader 
> > to local to see what is going on.
> >
> > One shard is fine but the other has a replica with has about 600MB of data 
> > to replay and it is extremely slow. Using the VisualVM sampler i find that 
> > the replayer is also spending almost all time in dealing with Java regex 
> > (stack trace below). Is this to be expected? And what is it actually doing? 
> > Where do the TokenFilters come from?
> >
> > I had a old but clean collection on the same cluster and started indexing 
> > to it to see what is going on but it too timed out due to Java regex. This 
> > is weird, because locally i have no problem indexing a million records in a 
> > 8.5.1 collection, and the broken down cluster has been running fine for 
> > over a month.
> >
> > A note, this index uses PreAnalyzedField, so i would expect no analysis or 
> > whatsoever, certainly no regex.
> >
> > Thanks,
> > Markus
> >
> > "replayUpdatesExecutor-3-thread-1-processing-n:127.0.1.1:8983_solr 
> > x:sitesearch_shard2_replica_t2 c:sitesearch s:shard2 r:core_node4" #222 
> > prio=5 os_prio=0 cpu=239207,44ms elapsed=239,50s tid=0x7ffde0057000 
> > nid=0x24f5 runnable  [0x7ffeedd0f000]
> >java.lang.Thread.State: RUNNABLE
> > at 
> > java.util.regex.Pattern$GroupTail.match(java.base@11.0.7/Pattern.java:4863)
> > at 
> > java.util.regex.Pattern$CharPropertyGreedy.match(java.base@11.0.7/Pattern.java:4306)
> > at 
> > java.util.regex.Pattern$GroupHead.match(java.base@11.0.7/Pattern.java:4804)
> > at 
> > java.util.regex.Pattern$CharPropertyGreedy.match(java.base@11.0.7/Pattern.java:4306)
> > at 
> > java.util.regex.Pattern$Start.match(java.base@11.0.7/Pattern.java:3619)
> > at 
> > java.util.regex.Matcher.search(java.base@11.0.7/Matcher.java:1729)
> > at java.util.regex.Matcher.find(java.base@11.0.7/Matcher.java:746)
> > at 
> > org.apache.lucene.analysis.pattern.PatternReplaceFilter.incrementToken(PatternReplaceFilter.java:71)
> > at 
> > org.apache.lucene.analysis.miscellaneous.TrimFilter.incrementToken(TrimFilter.java:42)
> > at 
> > org.apache.lucene.analysis.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:49)
> > at 
> > org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:812)
> > at 
> > org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
> > at 
> > org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
> > at 
> > org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
> > at 
> > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
> > at 
> > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> > at 
> > org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> > at 
> > org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:979)
> > at 
> > org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:345)
> > at 
> > org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:292)
> > at 
> > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:239)
> > at 
> > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
> > at 
> > org.apache.solr.update.

Re:

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 9:16 AM Nikolai Efseaff  wrote:
>
>
>
>
> Any tax advice in this e-mail should be considered in the context of the tax 
> services we are providing to you. Preliminary tax advice should not be relied 
> upon and may be insufficient for penalty protection.
> 
> The information contained in this message may be privileged and confidential 
> and protected from disclosure. If the reader of this message is not the 
> intended recipient, or an employee or agent responsible for delivering this 
> message to the intended recipient, you are hereby notified that any 
> dissemination, distribution or copying of this communication is strictly 
> prohibited. If you have received this communication in error, please notify 
> us immediately by replying to the message and deleting it from your computer.
>
> Notice required by law: This e-mail may constitute an advertisement or 
> solicitation under U.S. law, if its primary purpose is to advertise or 
> promote a commercial product or service. You may choose not to receive 
> advertising and promotional messages from Ernst & Young LLP (except for EY 
> Client Portal and the ey.com website, which track e-mail preferences through 
> a separate process) at this e-mail address by forwarding this message to 
> no-more-m...@ey.com. If you do so, the sender of this message will be 
> notified promptly. Our principal postal address is 5 Times Square, New York, 
> NY 10036. Thank you. Ernst & Young LLP


Re: velocity reponse writer javascript execution problem

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 7:32 AM Serkan KAZANCI  wrote:
>
> Hi,
>
>
>
> This is my first mail to the group. Nice to be here.
>
>
>
> 4 years ago, I have set up a solr search interface using velocity response
> writer templates. (Solr version : 5.3.1)
>
>
>
> I want to re-do the interface with new solr version(8.5.1). After some
> tests, I have realized that velocity response writer templates do not run
> JavaScript codes. Even the auto-complete feature at Solr's techproducts demo
> is not working, which also uses velocity response writer templates and
> relies on JavaScript for that function.
>
>
>
> Is it due to security vulnerability I have heard couple of years ago? Is
> there a work around so that I can use velocity templates that executes
> JavaScript? Or is it only me having this problem.
>
>
>
> Thanks for the replies in advance.
>
>
>
> Serkan,
>
>
>


Re: How to add MoreLikeThis MLT handler in Solr Cloud

2020-05-12 Thread Vignan Malyala
Any reference on this Is it possible actually?

On Tue, May 12, 2020 at 2:21 PM Vignan Malyala  wrote:

> How to add mlt handler in Solr Cloud?
>
> There is very limited documentation on this. Using search component with
> mlt=true doesn't include all configurations like boosting and mlt filters.
> Also the results with filters don't seem to work.
> Adding mlt handler seem better, but how to add in solr cloud.
> In standalone solr its easy to add mlt handler which we did, but what
> about solr cloud?
>
> Thanks in advance!
> Regards,
> Sai Vignan M
>


Re: Integrate highlighting data within main search results

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 11:35 PM Kamal Kishore Aggarwal
 wrote:
>
> any update on this guys
>
> On Wed, May 6, 2020 at 3:39 PM Kamal Kishore Aggarwal 
> wrote:
>
> > Hi,
> >
> > I am using highlighting feature in solr 8.3 with default method. With
> > current behaviour, main search results and highlighted results are shown in
> > different blocks. Is there a way we can implemented highlighting within the
> > search main results, without having to return extra block for highlighting?
> >
> > I believe that due to performance factor(like default limit values for
> > hl.maxAnalyzedChars, hl.snippets, hl.fragsize) that highlight is returned
> > as separate component. But, if someone has written custom component to
> > integrate both, please share the steps. Also, please share the performance
> > of it.
> >
> > Regards
> >
> > Kamal Kishore
> >


Re: Integrate highlighting data within main search results

2020-05-12 Thread Kamal Kishore Aggarwal
any update on this guys

On Wed, May 6, 2020 at 3:39 PM Kamal Kishore Aggarwal 
wrote:

> Hi,
>
> I am using highlighting feature in solr 8.3 with default method. With
> current behaviour, main search results and highlighted results are shown in
> different blocks. Is there a way we can implemented highlighting within the
> search main results, without having to return extra block for highlighting?
>
> I believe that due to performance factor(like default limit values for
> hl.maxAnalyzedChars, hl.snippets, hl.fragsize) that highlight is returned
> as separate component. But, if someone has written custom component to
> integrate both, please share the steps. Also, please share the performance
> of it.
>
> Regards
>
> Kamal Kishore
>


Re: How to add MoreLikeThis MLT handler in Solr Cloud

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 12:59 PM Vignan Malyala  wrote:
>
> Anyone knows how to add mlt handler in solr cloud?
>
> On Tue, May 12, 2020 at 2:21 PM Vignan Malyala  wrote:
>
> > How to add mlt handler in Solr Cloud?
> >
> > There is very limited documentation on this. Using search component with
> > mlt=true doesn't include all configurations like boosting and mlt filters.
> > Also the results with filters don't seem to work.
> > Adding mlt handler seem better, but how to add in solr cloud.
> > In standalone solr its easy to add mlt handler which we did, but what
> > about solr cloud?
> >
> > Thanks in advance!
> > Regards,
> > Sai Vignan M
> >


Re: Solr currency function and asymmetric rates

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 5:39 PM Murray Johnston
 wrote:
>
> I have a question / potential bug.  The currency function created in 
> https://issues.apache.org/jira/browse/SOLR-4138 first converts the field to 
> the default currency before then converting to the currency requested as part 
> of the function.  When dealing with asymmetric rates, that leads to incorrect 
> conversions.  Is this intended?  If not, is it required that 
> CurrencyFieldType.getValueSource convert to default currency?
>
>
> Thanks,
>
>
> -Murray


Re: Solr 8.5.1 query timeAllowed exceeded throws exception

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 5:37 PM Phill Campbell
 wrote:
>
> Upon examining the Solr source code it appears that it was unable to even 
> make a connection in the time allowed.
> While the error message was a bit confusing, I do understand what it means.
>
>
> > On May 12, 2020, at 2:08 PM, Phill Campbell  
> > wrote:
> >
> >
> >
> > org.apache.solr.client.solrj.SolrServerException: Time allowed to handle 
> > this request exceeded:…
> >   at 
> > org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
> >   at 
> > org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1143)
> >   at 
> > org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
> >   at 
> > org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
> >   at 
> > org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
> >   at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1035)
> > ...
> >   at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
> >   at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
> >   at java.util.concurrent.FutureTask.run(FutureTask.java)
> >   at javax.swing.SwingWorker.run(SwingWorker.java:334)
> >   at 
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >   at 
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >   at java.lang.Thread.run(Thread.java:748)
> > Caused by: 
> > org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> > from server at http://10.156.112.50:10001/solr/BTS: 
> > java.lang.NullPointerException
> >
> >   at 
> > org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
> >   at 
> > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
> >   at 
> > org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
> >   at 
> > org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
> >   at 
> > org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
> >
> >
> > The timeAllowed is set to 8 seconds. I am using a StopWatch to verify that 
> > the round trip was greater than 8 seconds.
> >
> > Documentation states:
> >
> > timeAllowed Parameter
> > This parameter specifies the amount of time, in milliseconds, allowed for a 
> > search to complete. If this time expires before the search is complete, any 
> > partial results will be returned, but values such as numFound, facet 
> > counts, and result stats may not be accurate for the entire result set. In 
> > case of expiration, if omitHeader isn’t set to true the response header 
> > contains a special flag called partialResults.
> >
> > I do not believe I should be getting an exception.
> >
> > I am load testing so I am intentionally putting pressure on the system.
> >
> > Is this the correct behavior to throw an exception?
> >
> > Regards.
>


Re: Limiting random results set with facets.

2020-05-12 Thread ART GALLERY
check out the videos on this website TROO.TUBE don't be such a
sheep/zombie/loser/NPC. Much love!
https://troo.tube/videos/watch/aaa64864-52ee-4201-922f-41300032f219

On Tue, May 12, 2020 at 9:38 PM David Lukowski  wrote:
>
> Thanks Srijan,  2 queries is exactly the route I started going today.
>
> Query 1:
> http://mysolr-node:8080/solr/M2_content/select
> ?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
> &start=0
> &rows=100
> &fq=channelId:(2 1 3 78 34 35 7 72)
> &fq=date:([* TO 2020-05-12T03:59:59.999Z])
> &hl=false
> &fl=id
> &wt=json
> &sort=random_123456 desc
>
>
> Query 2:
> http://mysolr-node:8080/solr/M2_content/select
> ?q= id:(12345 2345 3456 4567...)
> &start=0
> &rows=30
> &facet=true
> &facet.field=channelId
> &f.channelId.facet.limit=10
> &f.channelId.facet.mincount=1
> &hl=false
> &fl=id, text, users
> &wt=json
> &sort=date desc
>
> Working well so far, but still not ideal.
>
> Thanks for the assist,
>
> David
>
> On Tue, May 12, 2020 at 7:31 PM Srijan  wrote:
>
> > I see what you mean now. You could use two queries - first would return 100
> > randomly sorted docs (no faceting) and the second with fq that includes the
> > ids of the returned 100 docs + faceting.
> >
> > On Tue, May 12, 2020 at 1:29 PM David Lukowski 
> > wrote:
> >
> > > Thanks for the offer of help, this doesn't really seem like what I'm
> > > looking for though, but I could be misunderstanding.  I'll try to state
> > it
> > > more clearly and include the query.
> > >
> > >
> > > -- This will give me back all the documents that have "lucky" in them in
> > > RANDOM sorted order.
> > >
> > > http://mysolr-node:8080/solr/M2_content/select
> > > ?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
> > > &start=0
> > > &rows=0
> > > &fq=channelId:(2 1 3 78 34 35 7 72)
> > > &fq=date:([* TO 2020-05-12T03:59:59.999Z])
> > > &facet=true
> > > &facet.field=channelId
> > > &f.channelId.facet.limit=10
> > > &f.channelId.facet.mincount=1
> > > &hl=false
> > > &fl=id
> > > &wt=json
> > > &sort=random_123456 desc
> > >
> > >   The issue is that I only want 100 random results.  Sure, I could limit
> > > the results returned to the first 100 by specifying &rows=100, but the
> > > facets would match the query totals and not the rows returned totals.
> > >
> > > RESULTS I HAVE:
> > > "response":{"numFound":377895,"start":0,"docs":[]
> > >   },
> > >   "facet_counts":{
> > > "facet_queries":{},
> > > "facet_fields":{
> > >   "documentType":[
> > > "78",374015,
> > > "3",3021,
> > > "2",736,
> > > "1",41,
> > > "34",41,
> > > "35",32,
> > > "72",8,
> > > "7",1]},
> > >
> > >
> > > RESULTS I WANT:
> > > "response":{"numFound":100,"start":0,"docs":[]
> > >   },
> > >   "facet_counts":{
> > > "facet_queries":{},
> > > "facet_fields":{
> > >   "documentType":[
> > > "78",68,
> > > "3",22,
> > > "2",10]},
> > >
> > > How would I formulate the above query to give me a specific number of
> > > random results with the correct facet counts?
> > >
> > > Thanks for looking,
> > > David
> > >
> > > On Mon, May 11, 2020 at 2:09 PM Srijan  wrote:
> > >
> > > > If you can tag your filter query, you can exclude it when faceting.
> > Your
> > > > results will honor the filter query and you will get the N results
> > back,
> > > > and since faceting will exclude the filter, it will still give you
> > facet
> > > > count for the base query.
> > > >
> > > >
> > > >
> > >
> > https://lucene.apache.org/solr/guide/8_5/faceting.html#tagging-and-excluding-filters
> > > >
> > > >
> > > > On Mon, May 11, 2020 at 3:36 PM David Lukowski <
> > david.lukow...@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > > I'm looking for a way if possible to run a query with random results,
> > > > where
> > > > > I limit the number of results I want back, yet still have the facets
> > > > > accurately reflect the results I'm searching.
> > > > >
> > > > > When I run a search I use a filter query to randomize the results
> > based
> > > > on
> > > > > a modulo of a random seed. This returns a results set with the
> > > associated
> > > > > facets for each documentType.
> > > > >
> > > > > "response":{"numFound":377895,"start":0,"docs":[]
> > > > >   },
> > > > >   "facet_counts":{
> > > > > "facet_queries":{},
> > > > > "facet_fields":{
> > > > >   "documentType":[
> > > > > "78",374015,
> > > > > "3",3021,
> > > > > "2",736,
> > > > > "1",41,
> > > > > "34",41,
> > > > > "35",32,
> > > > > "72",8,
> > > > > "7",1]},
> > > > >
> > > > > How do I limit the number of results returned to N and have the
> > facets
> > > > > accurately reflect the number of messages?  I cannot simply say
> > rows=N
> > > > > because the facets will always reflect the total numFound and not the
> > > > > limited results set I'm looking for.
> > > > >
> > > >
> > >
> >


Re: Limiting random results set with facets.

2020-05-12 Thread David Lukowski
Thanks Srijan,  2 queries is exactly the route I started going today.

Query 1:
http://mysolr-node:8080/solr/M2_content/select
?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
&start=0
&rows=100
&fq=channelId:(2 1 3 78 34 35 7 72)
&fq=date:([* TO 2020-05-12T03:59:59.999Z])
&hl=false
&fl=id
&wt=json
&sort=random_123456 desc


Query 2:
http://mysolr-node:8080/solr/M2_content/select
?q= id:(12345 2345 3456 4567...)
&start=0
&rows=30
&facet=true
&facet.field=channelId
&f.channelId.facet.limit=10
&f.channelId.facet.mincount=1
&hl=false
&fl=id, text, users
&wt=json
&sort=date desc

Working well so far, but still not ideal.

Thanks for the assist,

David

On Tue, May 12, 2020 at 7:31 PM Srijan  wrote:

> I see what you mean now. You could use two queries - first would return 100
> randomly sorted docs (no faceting) and the second with fq that includes the
> ids of the returned 100 docs + faceting.
>
> On Tue, May 12, 2020 at 1:29 PM David Lukowski 
> wrote:
>
> > Thanks for the offer of help, this doesn't really seem like what I'm
> > looking for though, but I could be misunderstanding.  I'll try to state
> it
> > more clearly and include the query.
> >
> >
> > -- This will give me back all the documents that have "lucky" in them in
> > RANDOM sorted order.
> >
> > http://mysolr-node:8080/solr/M2_content/select
> > ?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
> > &start=0
> > &rows=0
> > &fq=channelId:(2 1 3 78 34 35 7 72)
> > &fq=date:([* TO 2020-05-12T03:59:59.999Z])
> > &facet=true
> > &facet.field=channelId
> > &f.channelId.facet.limit=10
> > &f.channelId.facet.mincount=1
> > &hl=false
> > &fl=id
> > &wt=json
> > &sort=random_123456 desc
> >
> >   The issue is that I only want 100 random results.  Sure, I could limit
> > the results returned to the first 100 by specifying &rows=100, but the
> > facets would match the query totals and not the rows returned totals.
> >
> > RESULTS I HAVE:
> > "response":{"numFound":377895,"start":0,"docs":[]
> >   },
> >   "facet_counts":{
> > "facet_queries":{},
> > "facet_fields":{
> >   "documentType":[
> > "78",374015,
> > "3",3021,
> > "2",736,
> > "1",41,
> > "34",41,
> > "35",32,
> > "72",8,
> > "7",1]},
> >
> >
> > RESULTS I WANT:
> > "response":{"numFound":100,"start":0,"docs":[]
> >   },
> >   "facet_counts":{
> > "facet_queries":{},
> > "facet_fields":{
> >   "documentType":[
> > "78",68,
> > "3",22,
> > "2",10]},
> >
> > How would I formulate the above query to give me a specific number of
> > random results with the correct facet counts?
> >
> > Thanks for looking,
> > David
> >
> > On Mon, May 11, 2020 at 2:09 PM Srijan  wrote:
> >
> > > If you can tag your filter query, you can exclude it when faceting.
> Your
> > > results will honor the filter query and you will get the N results
> back,
> > > and since faceting will exclude the filter, it will still give you
> facet
> > > count for the base query.
> > >
> > >
> > >
> >
> https://lucene.apache.org/solr/guide/8_5/faceting.html#tagging-and-excluding-filters
> > >
> > >
> > > On Mon, May 11, 2020 at 3:36 PM David Lukowski <
> david.lukow...@gmail.com
> > >
> > > wrote:
> > >
> > > > I'm looking for a way if possible to run a query with random results,
> > > where
> > > > I limit the number of results I want back, yet still have the facets
> > > > accurately reflect the results I'm searching.
> > > >
> > > > When I run a search I use a filter query to randomize the results
> based
> > > on
> > > > a modulo of a random seed. This returns a results set with the
> > associated
> > > > facets for each documentType.
> > > >
> > > > "response":{"numFound":377895,"start":0,"docs":[]
> > > >   },
> > > >   "facet_counts":{
> > > > "facet_queries":{},
> > > > "facet_fields":{
> > > >   "documentType":[
> > > > "78",374015,
> > > > "3",3021,
> > > > "2",736,
> > > > "1",41,
> > > > "34",41,
> > > > "35",32,
> > > > "72",8,
> > > > "7",1]},
> > > >
> > > > How do I limit the number of results returned to N and have the
> facets
> > > > accurately reflect the number of messages?  I cannot simply say
> rows=N
> > > > because the facets will always reflect the total numFound and not the
> > > > limited results set I'm looking for.
> > > >
> > >
> >
>


Re: Limiting random results set with facets.

2020-05-12 Thread Srijan
I see what you mean now. You could use two queries - first would return 100
randomly sorted docs (no faceting) and the second with fq that includes the
ids of the returned 100 docs + faceting.

On Tue, May 12, 2020 at 1:29 PM David Lukowski 
wrote:

> Thanks for the offer of help, this doesn't really seem like what I'm
> looking for though, but I could be misunderstanding.  I'll try to state it
> more clearly and include the query.
>
>
> -- This will give me back all the documents that have "lucky" in them in
> RANDOM sorted order.
>
> http://mysolr-node:8080/solr/M2_content/select
> ?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
> &start=0
> &rows=0
> &fq=channelId:(2 1 3 78 34 35 7 72)
> &fq=date:([* TO 2020-05-12T03:59:59.999Z])
> &facet=true
> &facet.field=channelId
> &f.channelId.facet.limit=10
> &f.channelId.facet.mincount=1
> &hl=false
> &fl=id
> &wt=json
> &sort=random_123456 desc
>
>   The issue is that I only want 100 random results.  Sure, I could limit
> the results returned to the first 100 by specifying &rows=100, but the
> facets would match the query totals and not the rows returned totals.
>
> RESULTS I HAVE:
> "response":{"numFound":377895,"start":0,"docs":[]
>   },
>   "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
>   "documentType":[
> "78",374015,
> "3",3021,
> "2",736,
> "1",41,
> "34",41,
> "35",32,
> "72",8,
> "7",1]},
>
>
> RESULTS I WANT:
> "response":{"numFound":100,"start":0,"docs":[]
>   },
>   "facet_counts":{
> "facet_queries":{},
> "facet_fields":{
>   "documentType":[
> "78",68,
> "3",22,
> "2",10]},
>
> How would I formulate the above query to give me a specific number of
> random results with the correct facet counts?
>
> Thanks for looking,
> David
>
> On Mon, May 11, 2020 at 2:09 PM Srijan  wrote:
>
> > If you can tag your filter query, you can exclude it when faceting. Your
> > results will honor the filter query and you will get the N results back,
> > and since faceting will exclude the filter, it will still give you facet
> > count for the base query.
> >
> >
> >
> https://lucene.apache.org/solr/guide/8_5/faceting.html#tagging-and-excluding-filters
> >
> >
> > On Mon, May 11, 2020 at 3:36 PM David Lukowski  >
> > wrote:
> >
> > > I'm looking for a way if possible to run a query with random results,
> > where
> > > I limit the number of results I want back, yet still have the facets
> > > accurately reflect the results I'm searching.
> > >
> > > When I run a search I use a filter query to randomize the results based
> > on
> > > a modulo of a random seed. This returns a results set with the
> associated
> > > facets for each documentType.
> > >
> > > "response":{"numFound":377895,"start":0,"docs":[]
> > >   },
> > >   "facet_counts":{
> > > "facet_queries":{},
> > > "facet_fields":{
> > >   "documentType":[
> > > "78",374015,
> > > "3",3021,
> > > "2",736,
> > > "1",41,
> > > "34",41,
> > > "35",32,
> > > "72",8,
> > > "7",1]},
> > >
> > > How do I limit the number of results returned to N and have the facets
> > > accurately reflect the number of messages?  I cannot simply say rows=N
> > > because the facets will always reflect the total numFound and not the
> > > limited results set I'm looking for.
> > >
> >
>


Solr currency function and asymmetric rates

2020-05-12 Thread Murray Johnston
I have a question / potential bug.  The currency function created in 
https://issues.apache.org/jira/browse/SOLR-4138 first converts the field to the 
default currency before then converting to the currency requested as part of 
the function.  When dealing with asymmetric rates, that leads to incorrect 
conversions.  Is this intended?  If not, is it required that 
CurrencyFieldType.getValueSource convert to default currency?


Thanks,


-Murray


Re: Solr 8.5.1 query timeAllowed exceeded throws exception

2020-05-12 Thread Phill Campbell
Upon examining the Solr source code it appears that it was unable to even make 
a connection in the time allowed.
While the error message was a bit confusing, I do understand what it means.


> On May 12, 2020, at 2:08 PM, Phill Campbell  
> wrote:
> 
> 
> 
> org.apache.solr.client.solrj.SolrServerException: Time allowed to handle this 
> request exceeded:…
>   at 
> org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
>   at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1143)
>   at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
>   at 
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
>   at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1035)
> ...
>   at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
>   at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
>   at java.util.concurrent.FutureTask.run(FutureTask.java)
>   at javax.swing.SwingWorker.run(SwingWorker.java:334)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://10.156.112.50:10001/solr/BTS: 
> java.lang.NullPointerException
> 
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>   at 
> org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
>   at 
> org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
> 
> 
> The timeAllowed is set to 8 seconds. I am using a StopWatch to verify that 
> the round trip was greater than 8 seconds.
> 
> Documentation states:
> 
> timeAllowed Parameter
> This parameter specifies the amount of time, in milliseconds, allowed for a 
> search to complete. If this time expires before the search is complete, any 
> partial results will be returned, but values such as numFound, facet counts, 
> and result stats may not be accurate for the entire result set. In case of 
> expiration, if omitHeader isn’t set to true the response header contains a 
> special flag called partialResults.
> 
> I do not believe I should be getting an exception.
> 
> I am load testing so I am intentionally putting pressure on the system.
> 
> Is this the correct behavior to throw an exception?
> 
> Regards.



Solr 8.5.1 query timeAllowed exceeded throws exception

2020-05-12 Thread Phill Campbell



org.apache.solr.client.solrj.SolrServerException: Time allowed to handle this 
request exceeded:…
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:345)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1143)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
at 
org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:1035)
...
at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
at java.util.concurrent.FutureTask.run(FutureTask.java)
at javax.swing.SwingWorker.run(SwingWorker.java:334)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://10.156.112.50:10001/solr/BTS: 
java.lang.NullPointerException

at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:665)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:265)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at 
org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)


The timeAllowed is set to 8 seconds. I am using a StopWatch to verify that the 
round trip was greater than 8 seconds.

Documentation states:

timeAllowed Parameter
This parameter specifies the amount of time, in milliseconds, allowed for a 
search to complete. If this time expires before the search is complete, any 
partial results will be returned, but values such as numFound, facet counts, 
and result stats may not be accurate for the entire result set. In case of 
expiration, if omitHeader isn’t set to true the response header contains a 
special flag called partialResults.

I do not believe I should be getting an exception.

I am load testing so I am intentionally putting pressure on the system.

Is this the correct behavior to throw an exception?

Regards.

Re: How to add MoreLikeThis MLT handler in Solr Cloud

2020-05-12 Thread Vignan Malyala
Anyone knows how to add mlt handler in solr cloud?

On Tue, May 12, 2020 at 2:21 PM Vignan Malyala  wrote:

> How to add mlt handler in Solr Cloud?
>
> There is very limited documentation on this. Using search component with
> mlt=true doesn't include all configurations like boosting and mlt filters.
> Also the results with filters don't seem to work.
> Adding mlt handler seem better, but how to add in solr cloud.
> In standalone solr its easy to add mlt handler which we did, but what
> about solr cloud?
>
> Thanks in advance!
> Regards,
> Sai Vignan M
>


Re: Limiting random results set with facets.

2020-05-12 Thread David Lukowski
Thanks for the offer of help, this doesn't really seem like what I'm
looking for though, but I could be misunderstanding.  I'll try to state it
more clearly and include the query.


-- This will give me back all the documents that have "lucky" in them in
RANDOM sorted order.

http://mysolr-node:8080/solr/M2_content/select
?q=({!terms f='permissionFilterId'}10,49 AND docBody:(lucky))
&start=0
&rows=0
&fq=channelId:(2 1 3 78 34 35 7 72)
&fq=date:([* TO 2020-05-12T03:59:59.999Z])
&facet=true
&facet.field=channelId
&f.channelId.facet.limit=10
&f.channelId.facet.mincount=1
&hl=false
&fl=id
&wt=json
&sort=random_123456 desc

  The issue is that I only want 100 random results.  Sure, I could limit
the results returned to the first 100 by specifying &rows=100, but the
facets would match the query totals and not the rows returned totals.

RESULTS I HAVE:
"response":{"numFound":377895,"start":0,"docs":[]
  },
  "facet_counts":{
"facet_queries":{},
"facet_fields":{
  "documentType":[
"78",374015,
"3",3021,
"2",736,
"1",41,
"34",41,
"35",32,
"72",8,
"7",1]},


RESULTS I WANT:
"response":{"numFound":100,"start":0,"docs":[]
  },
  "facet_counts":{
"facet_queries":{},
"facet_fields":{
  "documentType":[
"78",68,
"3",22,
"2",10]},

How would I formulate the above query to give me a specific number of
random results with the correct facet counts?

Thanks for looking,
David

On Mon, May 11, 2020 at 2:09 PM Srijan  wrote:

> If you can tag your filter query, you can exclude it when faceting. Your
> results will honor the filter query and you will get the N results back,
> and since faceting will exclude the filter, it will still give you facet
> count for the base query.
>
>
> https://lucene.apache.org/solr/guide/8_5/faceting.html#tagging-and-excluding-filters
>
>
> On Mon, May 11, 2020 at 3:36 PM David Lukowski 
> wrote:
>
> > I'm looking for a way if possible to run a query with random results,
> where
> > I limit the number of results I want back, yet still have the facets
> > accurately reflect the results I'm searching.
> >
> > When I run a search I use a filter query to randomize the results based
> on
> > a modulo of a random seed. This returns a results set with the associated
> > facets for each documentType.
> >
> > "response":{"numFound":377895,"start":0,"docs":[]
> >   },
> >   "facet_counts":{
> > "facet_queries":{},
> > "facet_fields":{
> >   "documentType":[
> > "78",374015,
> > "3",3021,
> > "2",736,
> > "1",41,
> > "34",41,
> > "35",32,
> > "72",8,
> > "7",1]},
> >
> > How do I limit the number of results returned to N and have the facets
> > accurately reflect the number of messages?  I cannot simply say rows=N
> > because the facets will always reflect the total numFound and not the
> > limited results set I'm looking for.
> >
>


Rule-Based Auth - update not working

2020-05-12 Thread Isabelle Giguere
Hi;

I'm using Solr 8.5.0.

I'm having trouble setting up some permissions using the rule-based 
authorization plugin: 
https://lucene.apache.org/solr/guide/8_5/rule-based-authorization-plugin.html

I have 3 users: "admin", "search", and "indexer".

I have set permissions and user roles:
"permissions": [  {  "name": "all", "role": "admin", "index": 1  },
  { "name": "admin-luke", "collection": "*", "role": "luke", "index": 2, 
"path": "/admin/luke"  },
  { "name": "read", "role": "searching", "index": 3  },
  {  "name": "update", "role": "indexing", "index": 4 }],
"user-role": {  "admin": "admin",
  "search": ["searching","luke"],
  "indexer": "indexing"   }  }
Attached: full output of GET /admin/authorization

So why can't user "indexer" add anything in a collection ?  I always get HTTP 
403 Forbidden.
Using Postman, I click the checkbox to show the password, so I'm sure I typed 
the right one.

Note that user "search" can't use the /select handler either, as should be the 
case with permission to "read".   This user can, however, use the Luke handler, 
as the custom permission allows.

User "admin" can use any API.  So at least the predefined permission "all" does 
work.

Note that the collections were created before enabling authentication and 
authorization.  Could that be the cause of the permission issues ?

Thanks;

Isabelle Giguère
Computational Linguist & Java Developer
Linguiste informaticienne & développeur java




permissions.json
Description: permissions.json


solr-user@lucene.apache.org

2020-05-12 Thread Nikolai Efseaff



Any tax advice in this e-mail should be considered in the context of the tax 
services we are providing to you. Preliminary tax advice should not be relied 
upon and may be insufficient for penalty protection.

The information contained in this message may be privileged and confidential 
and protected from disclosure. If the reader of this message is not the 
intended recipient, or an employee or agent responsible for delivering this 
message to the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication is strictly 
prohibited. If you have received this communication in error, please notify us 
immediately by replying to the message and deleting it from your computer.

Notice required by law: This e-mail may constitute an advertisement or 
solicitation under U.S. law, if its primary purpose is to advertise or promote 
a commercial product or service. You may choose not to receive advertising and 
promotional messages from Ernst & Young LLP (except for EY Client Portal and 
the ey.com website, which track e-mail preferences through a separate process) 
at this e-mail address by forwarding this message to no-more-m...@ey.com. If 
you do so, the sender of this message will be notified promptly. Our principal 
postal address is 5 Times Square, New York, NY 10036. Thank you. Ernst & Young 
LLP


velocity reponse writer javascript execution problem

2020-05-12 Thread Serkan KAZANCI
Hi,

 

This is my first mail to the group. Nice to be here.

 

4 years ago, I have set up a solr search interface using velocity response
writer templates. (Solr version : 5.3.1)

 

I want to re-do the interface with new solr version(8.5.1). After some
tests, I have realized that velocity response writer templates do not run
JavaScript codes. Even the auto-complete feature at Solr's techproducts demo
is not working, which also uses velocity response writer templates and
relies on JavaScript for that function.

 

Is it due to security vulnerability I have heard couple of years ago? Is
there a work around so that I can use velocity templates that executes
JavaScript? Or is it only me having this problem.

 

Thanks for the replies in advance.

 

Serkan,

 



RE: 8.5.1 LogReplayer extremely slow

2020-05-12 Thread Markus Jelsma
I found the bastard, it was a freaky document that skrewed Solr over, indexing 
kept failing, passing documents between replica's times out, documents get 
reindexed and so the document (and others) end up in the transaction log (many 
times) and are eligible for reindexing. Reindexing and replaying of the 
transaction log both fail on that specific document. Recovery was also not 
possible due to time outs.

Although the original document [1] is a mess, Solr should have no difficulties 
ingesting it [2]. Any ideas what is going on? Ticket, if so, about what 
exactly? For the record, this is PreAnalyzed.

Many thanks,
Markus

[1] https://pastebin.com/1NqBdYCM
[2] https://www.openindex.io/export/do_not_index.xml

-Original message-
> From:Markus Jelsma 
> Sent: Monday 11th May 2020 18:43
> To: solr-user 
> Subject: 8.5.1 LogReplayer extremely slow
> 
> Hello,
> 
> Our main Solr text search collection broke down last night (search was still 
> working fine), every indexing action timed out with the Solr master spending 
> most of its time in Java regex. One shard has only one replica left for 
> queries and it stays like that. I have copied both shard's leader to local to 
> see what is going on.
> 
> One shard is fine but the other has a replica with has about 600MB of data to 
> replay and it is extremely slow. Using the VisualVM sampler i find that the 
> replayer is also spending almost all time in dealing with Java regex (stack 
> trace below). Is this to be expected? And what is it actually doing? Where do 
> the TokenFilters come from?
> 
> I had a old but clean collection on the same cluster and started indexing to 
> it to see what is going on but it too timed out due to Java regex. This is 
> weird, because locally i have no problem indexing a million records in a 
> 8.5.1 collection, and the broken down cluster has been running fine for over 
> a month.
> 
> A note, this index uses PreAnalyzedField, so i would expect no analysis or 
> whatsoever, certainly no regex.
> 
> Thanks,
> Markus
> 
> "replayUpdatesExecutor-3-thread-1-processing-n:127.0.1.1:8983_solr 
> x:sitesearch_shard2_replica_t2 c:sitesearch s:shard2 r:core_node4" #222 
> prio=5 os_prio=0 cpu=239207,44ms elapsed=239,50s tid=0x7ffde0057000 
> nid=0x24f5 runnable  [0x7ffeedd0f000]
>    java.lang.Thread.State: RUNNABLE
> at 
>java.util.regex.Pattern$GroupTail.match(java.base@11.0.7/Pattern.java:4863)
> at 
>java.util.regex.Pattern$CharPropertyGreedy.match(java.base@11.0.7/Pattern.java:4306)
> at 
>java.util.regex.Pattern$GroupHead.match(java.base@11.0.7/Pattern.java:4804)
> at 
>java.util.regex.Pattern$CharPropertyGreedy.match(java.base@11.0.7/Pattern.java:4306)
> at 
>java.util.regex.Pattern$Start.match(java.base@11.0.7/Pattern.java:3619)
> at java.util.regex.Matcher.search(java.base@11.0.7/Matcher.java:1729)
> at java.util.regex.Matcher.find(java.base@11.0.7/Matcher.java:746)
> at 
>org.apache.lucene.analysis.pattern.PatternReplaceFilter.incrementToken(PatternReplaceFilter.java:71)
> at 
>org.apache.lucene.analysis.miscellaneous.TrimFilter.incrementToken(TrimFilter.java:42)
> at 
>org.apache.lucene.analysis.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:49)
> at 
>org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:812)
> at 
>org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
> at 
>org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
> at 
>org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
> at 
>org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
> at 
>org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> at 
>org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> at 
>org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:979)
> at 
>org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:345)
> at 
>org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:292)
> at 
>org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:239)
> at 
>org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
> at 
>org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
> at 
>org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:259)
> at 
>org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:489)
> at 
>org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpda

How to add MoreLikeThis MLT handler in Solr Cloud

2020-05-12 Thread Vignan Malyala
How to add mlt handler in Solr Cloud?

There is very limited documentation on this. Using search component with
mlt=true doesn't include all configurations like boosting and mlt filters.
Also the results with filters don't seem to work.
Adding mlt handler seem better, but how to add in solr cloud.
In standalone solr its easy to add mlt handler which we did, but what about
solr cloud?

Thanks in advance!
Regards,
Sai Vignan M


Re: Creating 100000 dynamic fields in solr

2020-05-12 Thread Jan Høydahl
Note that my example is simplified. Both the parent and child docs need to have 
globally unique ‘id’ fields, and any field names used both in parent and child 
needs to have same fieldType in schema.
There was some plans to automatically generate IDs for child documents if they 
do not exist, but I think that is not yet done. Perhaps you can add the UUID 
processor for this purpose?

 
   id
 
Jan

> 12. mai 2020 kl. 07:03 skrev Vignan Malyala :
> 
> Thanks Jan! This helps a lot!
> 
> Sai Vignan Malyala
> 
> On Mon, May 11, 2020 at 5:07 PM Jan Høydahl  wrote:
> 
>> Sounds like you are looking for parent/child docs here, see
>> https://lucene.apache.org/solr/guide/8_5/indexing-nested-documents.html
>> 
>> {
>>"type": "user",
>>"name": "user1",
>>"products": [
>>{ "id": "prod_A", "cost": 50},
>>{ "id": "prod_B", "cost": 200},
>>{ "id": "prod_D", "cost": 25}
>>]
>> }
>> 
>> This will index 4 documents - one user document and three product-cost
>> child documents.
>> 
>> You can then search the child docs and return matching parents with e.g.
>> q=*:*&fq={!parent which="type:user"}((id:prod_A AND cost:[50 TO 100]) OR
>> (id:prod_D AND cost:[0 TO 40]))&fl=[child]
>> 
>> Hope this helps.
>> 
>> Jan
>> 
>>> 11. mai 2020 kl. 11:35 skrev Vignan Malyala :
>>> 
>>> I have around 1M products used by my clients.
>>> Client need a filter of these 1M products by their cost filters.
>>> 
>>> Just like:
>>> User1 has 5 products (A,B,C,D,E)
>>> User2 has 3 products (D,E,F)
>>> User3 has 10 products (A,B,C,H,I,J,K,L,M,N,O)
>>> 
>>> ...every customer has different sets.
>>> 
>>> Now they want to search users by filter of product costs:
>>> Product_A_cost :  50 TO 100
>>> Product_D_cost :  0 TO 40
>>> 
>>> it should return all the users who use products in this filter range.
>>> 
>>> As I have 1M products, do I need to create dynamic fields for all users
>>> with filed names as Product_A_cost and product_B_cost. etc to make a
>>> search by them? If I should, then I haveto create 1M dynamic fields
>>> Or is there any other way?
>>> 
>>> Hope I'm clear here!
>>> 
>>> 
>>> On Mon, May 11, 2020 at 1:47 PM Jan Høydahl 
>> wrote:
>>> 
 Sounds like an anti pattern. Can you explain what search problem you are
 trying to solve with this many unique fields?
 
 Jan Høydahl
 
> 11. mai 2020 kl. 07:51 skrev Vignan Malyala :
> 
> Hi
> Is it good idea to create 10 dynamic fields of time pint in solr?
> I have that many fields to search on actually which come upon based on
> users.
> 
> Thanks in advance!
> And I'm using Solr Cloud in real-time.
> 
> Regards,
> Sai Vignan M
 
>> 
>>