Re: Facet Range with Stats

[Chris Hostetter]

: I'd like to generate stats for the results of a facet range.
: For example, calculate the mean sold price over a range of months.
: Does anyone know how to do this?
: This Jira issue seems to indicate its not yet possible.
: [SOLR-6352] Let Stats Hang off of Range Facets - ASF JIRA

This is possible using the JSON Facet API...

https://lucene.apache.org/solr/guide/7_3/json-facet-api.html

This newer API syntax is more expressive and easier to add new syntax to, 
so it's unlikely you'll see more features like this added to the older 
faceting API -- and will geet less and less likely as time goes on.


-Hoss
http://www.lucidworks.com/

Re: Multi threading indexing

[Rahul Singh]

Can try to leverage Spark to index. Or Kafka Connect with SolR.

--
Rahul Singh
rahul.si...@anant.us

Anant Corporation

On May 14, 2018, 2:03 AM -0500, Mikhail Khludnev , wrote:
> A few years ago I provided server side concurrency "booster"
> https://issues.apache.org/jira/browse/SOLR-3585.
> But now, I'd rather suppose it's client-side (or ETL) duty.
>
> On Mon, May 14, 2018 at 6:39 AM, Raymond Xie  wrote:
>
> > Hello,
> >
> > I have a huge amount of data (TB level) to be indexed, I am wondering if
> > anyone can share your idea/code to do the multithreading indexing?
> >
> > **
> > *Sincerely yours,*
> >
> >
> > *Raymond*
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev

Re: SolrCloud

[Rahul Singh]

Having concurrent DIH for example from the same source on different cluster 
nodes may cause duplicate work. But yes the ZK is what distributes the conf.

--
Rahul Singh
rahul.si...@anant.us

Anant Corporation

On May 16, 2018, 4:55 AM -0500, Jon Morisi , wrote:
> Hi All,
> I'm looking for additional information on how to configure an encrypted 
> password for the DIH Configuration File, when using solrcloud:
> https://lucene.apache.org/solr/guide/6_6/uploading-structured-data-store-data-with-the-data-import-handler.html#configuring-the-dih-configuration-file
>
> Is this compatible with solrcloud / zookeeper?
> What values are to be used for encryptKeyFile when running in SolrCloud?
> Is this a reference to a local directory?
> Is this a reference to a zookeeper directory?
> Should I put the file in my collections zookeeper conf dir, using the file 
> name only?
>
> Thanks,
> Jon

Re: SOLR ISSUE

[Shawn Heisey]

On 5/16/2018 9:27 AM, Shah, Rimple (LNG-RDU) wrote:
> https://lucene.apache.org/solr/guide/7_2/aws-solrcloud-tutorial.html
> I am trying to follow these instructions for running SOLR on EC2. Somehow I 
> am getting this error each and every time when I try to access the dashboard. 
> Is anyone able to help in this?
>
> HTTP ERROR 404
> Problem accessing /solr/. Reason:
>
> Not Found
> Caused by:
> javax.servlet.UnavailableException: Error processing the request. 
> CoreContainer is either not initialized or shutting down.

That message is the clue.  To figure out why there's a problem with
CoreContainer, you're going to need to find and examine the solr.log file.

If looking at the logfile doesn't give you any information you can
understand, you'll probably need to share any ERROR or WARN logs in the
file with us so we can pursue it further.  Note that these messages are
frequently quite long, and we will need all of it, not just the first
line or two.  Some of the information can't be properly interpreted
without the Solr version, so include that too.

Having access to the entire logfile would give us the best chance at
determining what's wrong.  Because the mailing list tends to eat all
email attachments, you'll need to use a file sharing website or a paste
website and provide a URL for us to access the file.

If I had to guess what might be wrong, reading the instructions you're
following, I am betting that there is no file named solr.xml in the
defined solr home, and there is no solr.xml in zookeeper either.  You
might be able to get this working by copying solr.xml from the
server/solr directory in the installation location to the defined solr
home.  Alternately, you could copy the file to zookeeper:

https://lucene.apache.org/solr/guide/7_2/using-zookeeper-to-manage-configuration-files.html#preparing-zookeeper-before-first-cluster-start

Thanks,
Shawn

Re: Re[2]: Solr CPU usage

[Deepak Goel]

1. Are you using two VM's on the same machine?

2. Why are the CPU usage graphs different (during the same time interval
15.40-16.00)? The master and slave are on the same computer, right?

3. The cpu utilised in the method graph is very less as compared to the
time interval shown in the cpu usage graph. Are there any other processes
running on the computer?


I have not used visualvm so its a bit confusing for me (I will have to
download and try it out)



Deepak
"The greatness of a nation can be judged by the way its animals are
treated. Please stop cruelty to Animals, become a Vegan"

+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

Make In India : http://www.makeinindia.com/home

On Wed, May 16, 2018 at 8:28 PM, Александр Шестак <
shestakalexa...@mail.ru.invalid> wrote:

>
> Master/slave are working on single computer with Intel Core i5 3.2GHz (it
> is 4 cores).
>
> It is standard visualVm ui with CPU usage. I think that it show overall
> CPU usage (for all cores).
> >Среда, 16 мая 2018, 17:42 +03:00 от Deepak Goel :
> >
> >How many CPU's do you have in master/slave?
> >
> >The graphs shown by you are for 'a single CPU' or 'All the CPU'?
> >
> >
> >
> >Deepak
> >" The greatness of a nation can be judged by the way its animals are
> treated.  Please stop cruelty to Animals, become a Vegan"
> >
> >+91 73500 12833
> >deic...@gmail.com
> >
> >Facebook:  https://www.facebook.com/deicool
> >LinkedIn:  www.linkedin.com/in/deicool
> >
> >"Plant a Tree, Go Green"
> >
> >Make In India :  http://www.makeinindia.com/home
> >
> >On Wed, May 16, 2018 at 6:41 PM, Александр Шестак  <
> shestakalexa...@mail.ru.invalid > wrote:
> >>Hi, I have a question about unpredictable CPU usage by solr.
> >>We have recently migrated our application from Solr 4.6.1 to Solr 7.1.0.
> We use master/slave approach. And now we have noticed that CPU usage of
> master/slave in passive state (no request to Solr are performed) is none
> zero. When we use Solr 4.6.1 and use tomcat for Solr deploying then CPU
> usage is almost 0. With Solr 7.1.0 cpu usage is varying from 0% to 40% (it
> jumps every time).
> >>
> >>Is it normal behavior for Solr and jetty?
> >>I have tried to analyze this situation by Java VisualVM.
> >>Solr Master CPU usage looks in next way
> >>
> >>Master spend most of time in some jetty method
> >>
> >>Solr Slave CPU usage looks in next way
> >>
> >>
> >>
> >>
> >>
> >>All this screenshots were made when there is no any activity with solr
> performed (solr just was started and no requests to it were performed)
> >
>
>
>
>

SOLR ISSUE

[Shah, Rimple (LNG-RDU)]

Hi,

https://lucene.apache.org/solr/guide/7_2/aws-solrcloud-tutorial.html
I am trying to follow these instructions for running SOLR on EC2. Somehow I am 
getting this error each and every time when I try to access the dashboard. Is 
anyone able to help in this?

HTTP ERROR 404
Problem accessing /solr/. Reason:

Not Found
Caused by:
javax.servlet.UnavailableException: Error processing the request. CoreContainer 
is either not initialized or shutting down.
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:342)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at 
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)

Thanks.
Rimple

Re[2]: Solr CPU usage

[Александр Шестак]

Master/slave are working on single computer with Intel Core i5 3.2GHz (it is 4 
cores).

It is standard visualVm ui with CPU usage. I think that it show overall CPU 
usage (for all cores). 
>Среда, 16 мая 2018, 17:42 +03:00 от Deepak Goel :
>
>How many CPU's do you have in master/slave?
>
>The graphs shown by you are for 'a single CPU' or 'All the CPU'?
>
>
>
>Deepak
>" The greatness of a nation can be judged by the way its animals are treated.  
>Please stop cruelty to Animals, become a Vegan"
>
>+91 73500 12833
>deic...@gmail.com
>
>Facebook:  https://www.facebook.com/deicool
>LinkedIn:  www.linkedin.com/in/deicool
>
>"Plant a Tree, Go Green"
>
>Make In India :  http://www.makeinindia.com/home
>
>On Wed, May 16, 2018 at 6:41 PM, Александр Шестак  < 
>shestakalexa...@mail.ru.invalid > wrote:
>>Hi, I have a question about unpredictable CPU usage by solr.
>>We have recently migrated our application from Solr 4.6.1 to Solr 7.1.0. We 
>>use master/slave approach. And now we have noticed that CPU usage of 
>>master/slave in passive state (no request to Solr are performed) is none 
>>zero. When we use Solr 4.6.1 and use tomcat for Solr deploying then CPU usage 
>>is almost 0. With Solr 7.1.0 cpu usage is varying from 0% to 40% (it jumps 
>>every time).
>>
>>Is it normal behavior for Solr and jetty?
>>I have tried to analyze this situation by Java VisualVM.
>>Solr Master CPU usage looks in next way
>>
>>Master spend most of time in some jetty method
>>
>>Solr Slave CPU usage looks in next way
>>
>>
>>
>>
>>
>>All this screenshots were made when there is no any activity with solr 
>>performed (solr just was started and no requests to it were performed)
>

Re: Solr CPU usage

[Deepak Goel]

How many CPU's do you have in master/slave?

The graphs shown by you are for 'a single CPU' or 'All the CPU'?



Deepak
"The greatness of a nation can be judged by the way its animals are
treated. Please stop cruelty to Animals, become a Vegan"

+91 73500 12833
deic...@gmail.com

Facebook: https://www.facebook.com/deicool
LinkedIn: www.linkedin.com/in/deicool

"Plant a Tree, Go Green"

Make In India : http://www.makeinindia.com/home

On Wed, May 16, 2018 at 6:41 PM, Александр Шестак <
shestakalexa...@mail.ru.invalid> wrote:

> Hi, I have a question about unpredictable CPU usage by solr.
>
> We have recently migrated our application from Solr 4.6.1 to Solr 7.1.0.
> We use master/slave approach. And now we have noticed that CPU usage of
> master/slave in passive state (no request to Solr are performed) is none
> zero. When we use Solr 4.6.1 and use tomcat for Solr deploying then CPU
> usage is almost 0. With Solr 7.1.0 cpu usage is varying from 0% to 40% (it
> jumps every time).
>
>
> Is it normal behavior for Solr and jetty?
>
> I have tried to analyze this situation by Java VisualVM.
>
> Solr Master CPU usage looks in next way
>
>
> Master spend most of time in some jetty method
>
>
> Solr Slave CPU usage looks in next way
>
>
>
>
>
>
> All this screenshots were made when there is no any activity with solr
> performed (solr just was started and no requests to it were performed)
>
>

Re: Solr CPU usage

[Shawn Heisey]

On 5/16/2018 7:11 AM, Александр Шестак wrote:


Hi, I have a question about unpredictable CPU usage by solr.

We have recently migrated our application from Solr 4.6.1 to Solr 
7.1.0. We use master/slave approach. And now we have noticed that CPU 
usage of master/slave in passive state (no request to Solr are 
performed) is none zero. When we use Solr 4.6.1 and use tomcat for 
Solr deploying then CPU usage is almost 0. With Solr 7.1.0 cpu usage 
is varying from 0% to 40% (it jumps every time).




The information you've obtained about where the CPU time is spent shows 
that your master server is spending its CPU time in Jetty code, not Solr 
code.  I do not know why.  Because it's code not written by this 
project, there may not be much we can do about it, other than encourage 
you to upgrade Solr so that you have a newer version of Jetty.  Version 
7.1.0 has Jetty 9.3.20, but 7.3.0 and 7.3.1 have Jetty 9.4.8.


The slave appears to be spending time in Lucene's merge code.  Which is 
very odd ... merging should normally happen on the master, usually not 
on slaves.


Thanks,
Shawn

Re: Solr facet on facet field returns junk values

[Mikhail Khludnev]

https://lucene.apache.org/solr/guide/6_6/faceting.html#Faceting-Thefacet.mincountParameter

On Wed, May 16, 2018 at 1:41 PM, Srinivas Kashyap <
srini...@tradestonesoftware.com> wrote:

> Hello,
>
> I have a Solr collection which has around 20 fields(indexed and stored).
> When I turn on facet and mention a facet.field, I'm able to get the facet
> count for that field. However, I'm able to see some junk facet counts like
> below, being generated in the facet response(1, 10, 100, 1000, 10
> ...). These junk values have count 0. However, when I query(*:*) and
> turn on facet.field, these junk values doesn't show up since the actual
> count values would be more. Any reason why is it happening?
>
> "PHY_KEY1": [
> "85",
> 11,
> "400",
> 10,
> "218",
> 9,
> "965",
> 9,
> "640",
> 5,
> "26",
> 3,
> "465",
> 3,
> "292",
> 2,
> "158",
> 1,
> "267",
> 1,
> "38",
> 1,
> "00176",
> 0,
> "1",
> 0,
> "10",
> 0,
> "100",
> 0,
> "1000",
> 0,
> "10",
> 0,
> "101",
> 0,
> "103",
> 0,
> "104",
> 0,
> "105",
> 0,
> "107",
> 0,
> "108",
> 0,
> "109",
> 0,
> "11",
> 0,
> "110",
> 0,
> "115",
> 0,
> "117",
> 0,
> "12",
> 0,
> "120",
> 0,
> "122",
> 0,
> "123",
> 0,
> "124",
> 0,
> "125",
> 0,
> "126",
> 0,
> "127",
>
>
> Thanks and Regards,
> Srinivas Kashyap
>



-- 
Sincerely yours
Mikhail Khludnev

Solr facet on facet field returns junk values

[Srinivas Kashyap]

Hello,

I have a Solr collection which has around 20 fields(indexed and stored). When I 
turn on facet and mention a facet.field, I'm able to get the facet count for 
that field. However, I'm able to see some junk facet counts like below, being 
generated in the facet response(1, 10, 100, 1000, 10 ...). These junk 
values have count 0. However, when I query(*:*) and turn on facet.field, these 
junk values doesn't show up since the actual count values would be more. Any 
reason why is it happening?

"PHY_KEY1": [
"85",
11,
"400",
10,
"218",
9,
"965",
9,
"640",
5,
"26",
3,
"465",
3,
"292",
2,
"158",
1,
"267",
1,
"38",
1,
"00176",
0,
"1",
0,
"10",
0,
"100",
0,
"1000",
0,
"10",
0,
"101",
0,
"103",
0,
"104",
0,
"105",
0,
"107",
0,
"108",
0,
"109",
0,
"11",
0,
"110",
0,
"115",
0,
"117",
0,
"12",
0,
"120",
0,
"122",
0,
"123",
0,
"124",
0,
"125",
0,
"126",
0,
"127",


Thanks and Regards,
Srinivas Kashyap

SolrCloud

[Jon Morisi]

Hi All,
I'm looking for additional information on how to configure an encrypted 
password for the DIH Configuration File, when using solrcloud:
https://lucene.apache.org/solr/guide/6_6/uploading-structured-data-store-data-with-the-data-import-handler.html#configuring-the-dih-configuration-file

Is this compatible with solrcloud / zookeeper?
What values are to be used for encryptKeyFile when running in SolrCloud?
Is this a reference to a local directory?
Is this a reference to a zookeeper directory?
Should I put the file in my collections zookeeper conf dir, using the file name 
only?

Thanks,
Jon

How to implement Solr auto suggester and spell checker simultaneously on a single search box

[Sonal Patwari]

Hello,

I am using Solr version 6.3 and I want to implement a search box with auto
suggester, spell checker and context filtering functionalities. I have
explored around search components '*suggest*' and '*spellcheck*', but I
found nothing which has all the 3 requirements fulfilled.


Thanks and Regards,
*Sonal Patwari* | Enterprise Software Engineer

[ANNOUNCE] Luke 7.3.1 released

[Tomoko Uchida]

Download the release zip here:
https://github.com/DmitryKey/luke/releases/tag/luke-7.3.1


- Upgrade to Lucene 7.3.1.
- UI Framework has been switched to JavaFX.

and, other changes in this release: 

>From this release, we will update JavaFX version only and stop maintenance
Thinlet Luke. Bug reports or feedback is welcome!

Note: If you use OpenJDK you need to install OpenJFX separately to run
JavaFX. Oracle HotSpot JVM includes JavaFX runtime.

-- 
Tomoko Uchida