Re: SUM Function performance

[Ganesh M]

All, Thanks for reply.

Regards,
Ganesh

On Sun 23 Oct, 2016 7:21 pm Yonik Seeley,  wrote:

> No reason to think it would be a problem.  10K documents isn't very much.
> -Yonik
>
>
> On Sun, Oct 23, 2016 at 3:14 AM, Ganesh M  wrote:
> > Is anyone tried summation of numeric field with 10k to 100k documents
> very frequently and faced any performance issues ?
> > Pls share your experience.
> >
> > On Sun 23 Oct, 2016 12:27 am Ganesh M, > wrote:
> > Hi,
> > We will have 10K documents for every hour. We would like to find sum on
> one field f1 based on certain condition and sum it based on group by
> another field f2
> > What will be the performance of it ? When this summation happens there
> could be other queries coming from other concurrent users.
> >
> > I am planning to do summing using following statement
> >
> > http://localhost:8983/solr/query?q=*:*;
> >json.facet={x:'sum(price)'}
> >
> > How far is this operation is costly. Can we execute this for every hour
> for 10k documents?
> >
> > Regards,
> > Ganesh
> >
>

Re: disable updates during startup

[Pushkar Raste]

The reason node is in recovery for long time could be related to
https://issues.apache.org/jira/browse/SOLR-9310

On Tue, Oct 4, 2016 at 9:14 PM, Rallavagu  wrote:

> Solr Cloud 5.4.1 with embedded Jetty - jdk 8
>
> Is there a way to disable incoming updates (from leader) during startup
> until "firstSearcher" queries finished? I am noticing that firstSearcher
> queries keep on running at the time of startup and node shows up as
> "Recovering".
>
> Thanks
>

Re: group.facet fails when facet on double field

[Pushkar Raste]

 This error is thrown when you add (or remove) on an existing field but do
not reindex you data from scratch. It is result of removing field cache
from Lucene. Although you were not getting error with Solr 4.8, I am pretty
sure that you were getting incorrect results.

Stand up a small test cluster with Solr 6.2.X and index a few documents in
it and  try your group.facet query, it would definitely work.

On Thu, Oct 20, 2016 at 9:18 AM, karel braeckman 
wrote:

> Hi,
>
> We are trying to upgrade from Solr 4.8 to Solr 6.2.
>
> This query:
>
> ?q=*%3A*=0=2=json=true=true&
> group.field=mediaObjectId=true=rating=true
>
> is returning the following error:
>
> null:org.apache.solr.common.SolrException: Exception during facet.field:
> rating
> at org.apache.solr.request.SimpleFacets.lambda$
> getFacetFieldCounts$0(SimpleFacets.java:739)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at org.apache.solr.request.SimpleFacets$2.execute(
> SimpleFacets.java:672)
> ...
> Caused by: java.lang.IllegalStateException: unexpected docvalues type
> NUMERIC for field 'mediaObjectId' (expected=SORTED). Re-index with
> correct docvalues type.
> at org.apache.lucene.index.DocValues.checkField(
> DocValues.java:212)
> at org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)
> at org.apache.lucene.search.grouping.term.
> TermGroupFacetCollector$SV.doSetNextReader(TermGroupFacetCollector.java:
> 128)
> ...
>
>
> The same query without the group.facet=true option does not give an
> error. On Solr 4.8 the query did not give problems.
>
>
> The relevant fields are configured as follows:
>
>
>  precisionStep="0" positionIncrementGap="0"/> type="double" indexed="true" stored="true" multiValued="false"
> /> multiValued="false" />
>
> Am I doing anything wrong, or do you have any suggestions on what to try
> next?
>
>
> Best regards
>
> Karel Braeckman
>

Re: SUM Function performance

[Yonik Seeley]

No reason to think it would be a problem.  10K documents isn't very much.
-Yonik


On Sun, Oct 23, 2016 at 3:14 AM, Ganesh M  wrote:
> Is anyone tried summation of numeric field with 10k to 100k documents very 
> frequently and faced any performance issues ?
> Pls share your experience.
>
> On Sun 23 Oct, 2016 12:27 am Ganesh M, 
> > wrote:
> Hi,
> We will have 10K documents for every hour. We would like to find sum on one 
> field f1 based on certain condition and sum it based on group by another 
> field f2
> What will be the performance of it ? When this summation happens there could 
> be other queries coming from other concurrent users.
>
> I am planning to do summing using following statement
>
> http://localhost:8983/solr/query?q=*:*;
>json.facet={x:'sum(price)'}
>
> How far is this operation is costly. Can we execute this for every hour for 
> 10k documents?
>
> Regards,
> Ganesh
>

Re: SUM Function performance

[Susheel Kumar]

Hi Ganesh,

In general it shouldn't be an issue if you execute sum queries every other
hour but you may want to share your cluster configuration (solr version,
solr cloud?, # machines, machine configuration, index size) and
load(indexing & query load) and perform some tests.

Also FYI, there is streaming/sql functionality available in latest Solr
(6.0 above) which are capable of performing these aggregations etc. in
parallel(map-reduce way).

Thanks,
Susheel

On Sun, Oct 23, 2016 at 3:14 AM, Ganesh M  wrote:

> Is anyone tried summation of numeric field with 10k to 100k documents very
> frequently and faced any performance issues ?
> Pls share your experience.
>
> On Sun 23 Oct, 2016 12:27 am Ganesh M,  s...@live.in>> wrote:
> Hi,
> We will have 10K documents for every hour. We would like to find sum on
> one field f1 based on certain condition and sum it based on group by
> another field f2
> What will be the performance of it ? When this summation happens there
> could be other queries coming from other concurrent users.
>
> I am planning to do summing using following statement
>
> http://localhost:8983/solr/query?q=*:*;
>json.facet={x:'sum(price)'}
>
> How far is this operation is costly. Can we execute this for every hour
> for 10k documents?
>
> Regards,
> Ganesh
>
>

Show Facet Aggregation Functions fields even when the result is 0

[Zheng Lin Edwin Yeo]

Hi,

I'm using JSON Facet in Solr 5.4.0. Currently, this is what I'm getting in
the JSON output when I'm using the Aggregation Functions.

"facets":{
"count":9,
"doDate_dts":{
  "buckets":[{
{
  "val":"2016-08-29T00:00:00Z",
  "count":3,
  "sum":10.3,
  "avg":3.5},
{
  "val":"2016-08-30T00:00:00Z",
  "count":0},
{
  "val":"2016-08-31T00:00:00Z",
  "count":0},
{
  "val":"2016-09-01T00:00:00Z",
  "count":7,
  "sum":18.9,
  "avg":5.6},
{
  "val":"2016-09-02T00:00:00Z",
  "count":0},

>From the JSON output, we can see that the Aggregation Functions ("sum" and
"avg") are only shown on the facet which has results. For those Facet with
count are 0, the Aggregation Functions ("sum" and "avg") are not shown.

Is it possible to show the Aggregation Functions ("sum" and "avg") on the
JSON output as well, even when the count is 0? We can just show all 0, like
the following which I have modified manually from the above actual output:

"facets":{
"count":9,
"doDate_dts":{
  "buckets":[{
{
  "val":"2016-08-29T00:00:00Z",
  "count":3,
  "sum":10.3,
  "avg":3.5},
{
  "val":"2016-08-30T00:00:00Z",
  "count":0,
  "sum":0,
  "avg":0},
{
  "val":"2016-08-31T00:00:00Z",
  "count":0,
  "sum":0,
  "avg":0},
{
  "val":"2016-09-01T00:00:00Z",
  "count":7,
  "sum":18.9,
  "avg":5.6},
{
  "val":"2016-09-02T00:00:00Z",
  "count":0,
  "sum":0,
  "avg":0},

Regards,
Edwin

Re: SUM Function performance

[Ganesh M]

Is anyone tried summation of numeric field with 10k to 100k documents very 
frequently and faced any performance issues ?
Pls share your experience.

On Sun 23 Oct, 2016 12:27 am Ganesh M, 
> wrote:
Hi,
We will have 10K documents for every hour. We would like to find sum on one 
field f1 based on certain condition and sum it based on group by another field 
f2
What will be the performance of it ? When this summation happens there could be 
other queries coming from other concurrent users.

I am planning to do summing using following statement

http://localhost:8983/solr/query?q=*:*;
   json.facet={x:'sum(price)'}

How far is this operation is costly. Can we execute this for every hour for 10k 
documents?

Regards,
Ganesh