Error getting replica locations : unable to get autoscaling policy session when creating collection

[Zheng Lin Edwin Yeo]

Hi,

I am using Solr 7.6.0.

When I tried to create a collection, I encountered the error of unable to
get autoscaling policy session.

I have 2 replicas running, and I am creating it with the following command:
https://localhost:8983/solr/admin/collections?action=CREATE=collection1=1=2=2=collection1

What could be the reason that caused this error?

This is the error which I am getting:

Saturday, February 2, 2019 12:40:08 PM ERROR true
OverseerCollectionMessageHandler Collection: calls operation: create
failed:org.apache.solr.common.SolrException: Error getting replica
locations : unable to get autoscaling policy session

Collection: calls operation: create
failed:org.apache.solr.common.SolrException: Error getting replica
locations : unable to get autoscaling policy session

 at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:183)

 at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:259)

 at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:478)

 at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)

 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

 at java.lang.Thread.run(Unknown Source)

Caused by: org.apache.solr.common.SolrException: unable to get autoscaling
policy session

 at
org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper.getReplicaLocations(PolicyHelper.java:123)

 at
org.apache.solr.cloud.api.collections.Assign.getPositionsUsingPolicy(Assign.java:369)

 at
org.apache.solr.cloud.api.collections.Assign$PolicyBasedAssignStrategy.assign(Assign.java:617)

 at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.buildReplicaPositions(CreateCollectionCmd.java:394)

 at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:178)

 ... 6 more

Caused by: org.apache.solr.common.SolrException:
org.apache.solr.common.SolrException:

 at
org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:78)

 at
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchTagValues(SolrClientNodeStateProvider.java:138)

 at
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getNodeValues(SolrClientNodeStateProvider.java:127)

 at
org.apache.solr.client.solrj.cloud.autoscaling.Row.(Row.java:66)

 at
org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.(Policy.java:566)

 at
org.apache.solr.client.solrj.cloud.autoscaling.Policy.createSession(Policy.java:391)

 at
org.apache.solr.client.solrj.cloud.autoscaling.Policy.createSession(Policy.java:353)

 at
org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper$SessionRef.createSession(PolicyHelper.java:460)

 at
org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper$SessionRef.get(PolicyHelper.java:425)

 at
org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper.getSession(PolicyHelper.java:481)

 at
org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper.getReplicaLocations(PolicyHelper.java:121)

 ... 10 more

Caused by: org.apache.solr.common.SolrException:

 at
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$AutoScalingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:301)

 at
org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:76)

 ... 20 more

Caused by: org.apache.solr.client.solrj.SolrServerException: IOException
occured when talking to server at: http://10.33.42.58:8983/edm

 at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:657)

 at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)

 at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)

 at
org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1260)

 at
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:349)

 at
org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$AutoScalingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:274)

 ... 21 more

Caused by: org.apache.http.client.ClientProtocolException

 at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:187)

 at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)

 at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)

 at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)

 ... 26 more

Caused by: org.apache.http.ProtocolException: The server failed to respond
with a valid HTTP response

 at

Re: Using the terms component in Solr Cloud gives random result

[Markus Kalkbrenner]

I’ll answer my own question:

setting distrib=true solved the issue … mostly

Our client requests wt=json =flat

But the result isn’t flat!

As soon I set distrib=true the json response is formatted as a map instead go 
being flat.

Did I find a bug or this a known limitation?

Markus

> Am 31.01.2019 um 19:43 schrieb Markus Kalkbrenner 
> :
> 
> Hi,
> 
> I tried to use the terms component with the techproducts example in cloud 
> mode and was surprised, that the results toggle for each request.
> 
> The response to
> http://localhost:8983/solr/techproducts/terms?terms=true=name
> permanently toggles between these two results:
> 
> {
>  "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":0},
>  "terms":{
>"name":[
>  "184",3,
>  "1gb",3,
>  "3200",3,
>  "400",3,
>  "ddr",3,
>  "memory",3,
>  "pc",3,
>  "pin",3,
>  "sdram",3,
>  "system",3]}}
> 
> {
>  "responseHeader":{
>"zkConnected":true,
>"status":0,
>"QTime":0},
>  "terms":{
>"name":[
>  "one",3,
>  "canon",2,
>  "drive",2,
>  "gb",2,
>  "hard",2,
>  "11",1,
>  "133",1,
>  "19",1,
>  "250",1,
>  "300",1]}}
> 
> 
> To reproduce the issue, here’s how I setup the test environment:
> 
> bin/solr start -e cloud -noprompt
> 
> bin/solr create -c techproducts -s 2 -rf 2 -d 
> server/solr/configsets/sample_techproducts_configs/conf -n 
> sample_techproducts_configs
> 
> bin/post -c techproducts example/exampledocs/*.xml
> 
> 
> In the admin UI I can see that 18 documents are indexed in one core and 14 in 
> the other. (The techproducts example contains 32 docs in total.)
> 
> Obviously I expected to get the terms for all documents stored in the 
> collection and not randomly from one core or the other.
> 
> Am I doing something wrong?
> 
> Regards,
> Markus
> 
> 
> 

Markus Kalkbrenner
Dipl.-Ing. (FH) techn. Informatik
CTO

T +49 69 / 348 739 110

markus.kalkbren...@biologis.com


bio.logis Genetic Information Management GmbH
Zentrale:
Olof-Palme-Str. 15
D-60439 Frankfurt am Main

Geschäftsführung: Prof. Dr. med. Daniela Steinberger, Dipl. Betriebswirt Enrico 
Just
Firmensitz: Altenhöferallee 3, 60438 Frankfurt am Main
Registergericht Frankfurt am Main, HRB 97945

Re: Query over nested documents with an AND Operator

[Scott Stults]

Hi Julia,

Keep in mind that in order to facet on child document fields you'll need to
use the block join facet component:
https://lucene.apache.org/solr/guide/7_4/blockjoin-faceting.html

For the query itself you probably need to specify each required attribute
value, but looks like you're already heading down that path with the
facets. Add required local queries wrapped in the default query parser. The
local queries themselves would be block joins similar to this:

"+{!parent which=contenttype_s:parentDocument}attributevalue_s:brass
+{!parent which=contenttype_s:parentDocument}attributevalue_s:plastic"

That requires that a parent document satisfies both child document
constraints.

Also, if you want to return the child documents you'll need to use the
ChildDocTransformerFactory:
"fl=id,[child parentFilter=contenttype_s:parentDocument]"
(I'm not sure if that's required if you just want to facet on the child doc
values and not display the other fields.)

Hope that helps!

-Scott


On Fri, Feb 1, 2019 at 8:51 AM Mikhail Khludnev  wrote:

> Whats' your current query? It's probably a question of building boolean
> query by combining Solr queries.
> Note, this datamodel might be a little bit overwhelming, So, if number of
> distinct attributename values is around a thousand, just handle it via
> dynamic field without nesting docs:
>
>
>   brass
>
> 1
> >
> >   4711
> >
> >   here is a short text dealing with plastic and
> > brass
> >
> >   here is a detailed description
> >
> >   parentDocument
> >
> >   
> >
> > 
> >
> >   2
> >
> >   4811
> >
> >   here is a shorttext
> >
> >   here you will find a detailed
> description
> >
> >   parentDocument
> >
> >   
> >
> > 
> >
> >   2_1
> >
> >   material 
> >
> >   brass
> >
> >   
> >
> >   
> >
> >   2_2
> >
> >   material quality
> >
> >   plastic
> >
> >   
> >
> > 
> >
> > 
> >
> > I need an AND operator between my queries because I want to get as
> > accurate hits as possible. I managed to search all Parent and Child
> > Documents with one search term and get the right result.
> >
> > But if I want to search for example for plastic and brass (that means 2
> or
> > more search terms). I want to get both the Parent Document for the
> > respective child document as result (article 4811), as well as article
> 4711
> > because in this article the two words appear in the description. But the
> > result of my query is always only article 4711. I know that I could also
> > write the attribute in one field. However, I want to have a facet about
> the
> > attribute name.
> >
> >
> >
> > I hope you can help me with this problem.
> >
> >
> >
> > Thank you very much,
> >
> >
> >
> > Mit freundlichen Grüßen / Kind regards
> >
> >
> > *Julia Gelszus *
> > Bachelor of Science
> > Consultant SAP Development Workbench
> >
> >
> > *FIS Informationssysteme und Consulting GmbH *Röthleiner Weg 1
> > 97506 Grafenrheinfeld
> >
> > P +49 (9723) 9188-667
> > F +49 (9723) 9188-200
> > E j.gels...@fis-gmbh.de
> > www.fis-gmbh.de
> >
> > Managing Directors:
> > Ralf Bernhardt, Wolfgang Ebner, Frank Schöngarth
> >
> > Registration Office Schweinfurt HRB 2209
> >
> >   
> > 
> > 
> > 
> >
> >
>
>
> --
> Sincerely yours
> Mikhail Khludnev
>


-- 
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com

RE: Need to perfom search and group the record on basis of domain,subject,from address and display the count of label i.e inbox,spam

[Margaret Owens]

Please remove me from this list.

-Original Message-
From: Scott Stults  
Sent: February 1, 2019 2:03 PM
To: solr-user@lucene.apache.org
Subject: Re: Need to perfom search and group the record on basis of 
domain,subject,from address and display the count of label i.e inbox,spam

Hi Swapnil,

There wasn't a a question in your post, so I'm guessing you're having trouble 
getting started. Take a look at the JSON Facet API. That should get you most of 
the way there.

https://lucene.apache.org/solr/guide/7_5/json-facet-api.html

k/r,
Scott

On Fri, Feb 1, 2019 at 7:36 AM swap  wrote:

> Need to perfom search and group the record on basis of 
> domain,subject,from address and display the count of label i.e inbox,spam
>   and label status i.e read and unread with it.The label and label 
> status should be displayed as percentage.
>
> Scenorio 1
> Document structure is as mentioned below indexed in solr. message_id 
> is unique field in solr
>   {
> "email_date_time": 1548922689,
> "subject": "abcdef",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "123456789ui",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "g...@gmail.com",
> "label_status": "unread"
>   }
>
>   {
> "email_date_time": 1548922689,
> "subject": "abcdef",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "zxiu22",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "g...@gmail.com",
> "label_status": "unread"
>   }
>
>   {
> "email_date_time": 1548922689,
> "subject": "defg",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "ftyuiooo899",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "f...@gmail.com",
> "label_status": "unread"
>   }
>
> I have below mentioned point to be implemented
>
> 1. Need to perfom search and group the record on basis of 
> domain,subject,from address and display the count of label i.e inbox,spam
>   and label status i.e read and unread with it.The label and label 
> status should be displayed as percentage.
>
>
> 2. Need to paginate the record along with the implementation 1
>
>
> Display will be as mentioned below
>
>
> 1. domain name : @ subject:hello from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80% read percentage  : 30%  
> unread percentage : 70%
>
> 2. domain name : @ subject:hi from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80% read percentage  : 30%  
> unread percentage : 70%
>
>
> 3. domain name : @ subject:where from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80% read percentage  : 30%  
> unread percentage : 70%
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


--
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com

Re: Need to perfom search and group the record on basis of domain,subject,from address and display the count of label i.e inbox,spam

[Scott Stults]

Hi Swapnil,

There wasn't a a question in your post, so I'm guessing you're having
trouble getting started. Take a look at the JSON Facet API. That should get
you most of the way there.

https://lucene.apache.org/solr/guide/7_5/json-facet-api.html

k/r,
Scott

On Fri, Feb 1, 2019 at 7:36 AM swap  wrote:

> Need to perfom search and group the record on basis of domain,subject,from
> address and display the count of label i.e inbox,spam
>   and label status i.e read and unread with it.The label and label status
> should be displayed as percentage.
>
> Scenorio 1
> Document structure is as mentioned below indexed in solr. message_id is
> unique field in solr
>   {
> "email_date_time": 1548922689,
> "subject": "abcdef",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "123456789ui",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "g...@gmail.com",
> "label_status": "unread"
>   }
>
>   {
> "email_date_time": 1548922689,
> "subject": "abcdef",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "zxiu22",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "g...@gmail.com",
> "label_status": "unread"
>   }
>
>   {
> "email_date_time": 1548922689,
> "subject": "defg",
> "created": 1548932108,
> "domain": ".com",
> "message_id": "ftyuiooo899",
> "label": "inbox",
> "from_address": xxxbc.com",
> "email": "f...@gmail.com",
> "label_status": "unread"
>   }
>
> I have below mentioned point to be implemented
>
> 1. Need to perfom search and group the record on basis of
> domain,subject,from address and display the count of label i.e inbox,spam
>   and label status i.e read and unread with it.The label and label status
> should be displayed as percentage.
>
>
> 2. Need to paginate the record along with the implementation 1
>
>
> Display will be as mentioned below
>
>
> 1. domain name : @ subject:hello from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80%
> read percentage  : 30%  unread percentage : 70%
>
> 2. domain name : @ subject:hi from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80%
> read percentage  : 30%  unread percentage : 70%
>
>
> 3. domain name : @ subject:where from addredd: abcd@i
>
> inbox percentage : 20% spam percentage : 80%
> read percentage  : 30%  unread percentage : 70%
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>


-- 
Scott Stults | Founder & Solutions Architect | OpenSource Connections, LLC
| 434.409.2780
http://www.opensourceconnections.com

Re: [CDCR]Unable to locate core

[Tim]

After some more investigation it seems that we're running into the  same bug
found here   .

However if my understanding is correct that bug in 7.3 was patched out.
Unfortunately we're running into the same behavior in 7.5

CDCR is replicating successfully to the leader node but is not replicating
to the followers.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

change in White Space when upgrading 6.6 to 7.4

[Oakley, Craig (NIH/NLM/NCBI) [C]]

We had a problem when upgrading from Solr 6.6 to Solr 7.4 in that a query 
ceased to work.


The query was of the form 
http://localhost:8983/solr/collection/select?indent=on=ABC4856.21%20AND%20-field1:ABC4856.21=json=0

Basically finding a count of those records where there is some field which has 
"ABC4856.21", but where the field field1 does not have that string (in other 
words, where there is some field other than field1 which has "ABC4856.21")

For this particular collection, running the query against Solr 6.6 resulted in 
"response":{"numFound":0" (which was correct), but running it against Solr 7.4 
resulted in ""response":{"numFound":21322074"

After some investigation, it seemed to be a problem with the initial 
"ABC4856.21" being tokenized as "ABC4856" and "21"

We found various work-arounds such as putting quotation marks around the string 
or adding "*:" after the "q="; but the user wanted the exact same query to work 
in Solr 7.4 as it had in Solr 6.6

Eventually, we found a solution by adding "true" to the 
Select handler in solrconfig.xml (for "Separate On Whitespace").

This solution seems to be sufficient; but we would like to be sure we 
understand the solution.

Looking at lucene.apache.org/solr/guide/7_4/tokenizers.html#standard-tokenizer 
it would seem that the period should not split the string into two tokens.

Could someone clarify how we can know which Tokenize is used when, and which 
definition of White Space is used when?

Thanks

Re: Query over nested documents with an AND Operator

[Mikhail Khludnev]

Whats' your current query? It's probably a question of building boolean
query by combining Solr queries.
Note, this datamodel might be a little bit overwhelming, So, if number of
distinct attributename values is around a thousand, just handle it via
dynamic field without nesting docs:


  brass

1
>
>   4711
>
>   here is a short text dealing with plastic and
> brass
>
>   here is a detailed description
>
>   parentDocument
>
>   
>
> 
>
>   2
>
>   4811
>
>   here is a shorttext
>
>   here you will find a detailed description
>
>   parentDocument
>
>   
>
> 
>
>   2_1
>
>   material 
>
>   brass
>
>   
>
>   
>
>   2_2
>
>   material quality
>
>   plastic
>
>   
>
> 
>
> 
>
> I need an AND operator between my queries because I want to get as
> accurate hits as possible. I managed to search all Parent and Child
> Documents with one search term and get the right result.
>
> But if I want to search for example for plastic and brass (that means 2 or
> more search terms). I want to get both the Parent Document for the
> respective child document as result (article 4811), as well as article 4711
> because in this article the two words appear in the description. But the
> result of my query is always only article 4711. I know that I could also
> write the attribute in one field. However, I want to have a facet about the
> attribute name.
>
>
>
> I hope you can help me with this problem.
>
>
>
> Thank you very much,
>
>
>
> Mit freundlichen Grüßen / Kind regards
>
>
> *Julia Gelszus *
> Bachelor of Science
> Consultant SAP Development Workbench
>
>
> *FIS Informationssysteme und Consulting GmbH *Röthleiner Weg 1
> 97506 Grafenrheinfeld
>
> P +49 (9723) 9188-667
> F +49 (9723) 9188-200
> E j.gels...@fis-gmbh.de
> www.fis-gmbh.de
>
> Managing Directors:
> Ralf Bernhardt, Wolfgang Ebner, Frank Schöngarth
>
> Registration Office Schweinfurt HRB 2209
>
>   
> 
> 
> 
>
>


-- 
Sincerely yours
Mikhail Khludnev

SolrJ removes header set by HttpClient

[Erlend Garåsen]

For some reasons, the "Expect: 100-continue" header will not be set,
even though it's configured in HttpClient:
builder.setDefaultRequestConfig(RequestConfig.custom().setExpectContinueEnabled(true).build());

A HttpClient developer has reviewed the code and says we're setting up
the client correctly, so I have a reason to believe there is a bug in
SolrJ. It's actually a problem we are facing in ManifoldCF, explained in:
https://issues.apache.org/jira/browse/CONNECTORS-1564

I have created a small application which can build by Maven in order to
test it. It logs the http traffic to missing-header.log. The application
also adds another header which *does* show up in the app log:
builder.setDefaultHeaders(Arrays.asList(new
BasicHeader("X-Default-Header", "default header")));

DEBUG 01.02.2019 13:14:59:768 (Wire.java:wire:73) - http-outgoing-0 >>
"X-Default-Header: default header[\r][\n]"

But no Expect: 100-continue header.

The small app/code example is available here:
http://folk.uio.no/erlendfg/solr/

Solr 7.4.0 is used in the app, but the same happens by using 7.6.0 (can
be configured in pom.xml).

It's actually as simple as this:
HttpClientBuilder builder = HttpClients.custom();
// This header shows up:
builder.setDefaultHeaders(Arrays.asList(new
BasicHeader("X-Default-Header", "default header")));
// This should add an Expect: 100-continue header, but it's missing:
 
builder.setDefaultRequestConfig(RequestConfig.custom().setExpectContinueEnabled(true).build());

HttpClient httpClient = builder.build();
HttpSolrClient client = new HttpSolrClient.Builder()
  .withHttpClient(httpClient)
  .withBaseSolrUrl("http://localhost:8983/solr;).build();

Erlend

Re: Solr Size Limitation upto 32 kb limitation

[Kranthi Kumar K]

Hi Team,


Thanks for your suggestions that you've posted, but none of them have fixed our 
issue. Could you please provide us your valuable suggestions to address this 
issue.


We'll be awaiting your reply.


Thanks,

Kranthi kumar.K


From: Michelle Ngo
Sent: Thursday, January 24, 2019 12:00:06 PM
To: Kranthi Kumar K; d...@lucene.apache.org; solr-user@lucene.apache.org
Cc: Ananda Babu medida; Srinivasa Reddy Karri; Ravi Vangala; Suresh Malladi; 
Vijay Nandula
Subject: RE: Solr Size Limitation upto 32 kb limitation


Thanks @Kranthi Kumar K for following up



From: Kranthi Kumar K 
Sent: Thursday, 24 January 2019 4:51 PM
To: d...@lucene.apache.org; solr-user@lucene.apache.org
Cc: Ananda Babu medida ; Srinivasa Reddy 
Karri ; Michelle Ngo 
; Ravi Vangala ; 
Suresh Malladi ; Vijay Nandula 

Subject: RE: Solr Size Limitation upto 32 kb limitation



Thank you Bernd Fehling for your suggested solution, I've tried the same by 
changing the type and added multivalued to true in Schema.xml file i.e,

change from:







Changed to:







After changing it also still we are unable to import the files size > 32 kb. 
please find the solution suggested by Bernd in the below url:



http://lucene.472066.n3.nabble.com/Re-Solr-Size-Limitation-upto-32-kb-limitation-td4421569.html



Bernd Fehling, could you please suggest another alternative solution to resolve 
our issue, which would help us alot?



Please let me know for any questions.



[image001]

Thanks & Regards,

Kranthi Kumar.K,

Software Engineer,

Ccube Fintech Global Services Pvt Ltd.,

Email/Skype: 
kranthikuma...@ccubefintech.com,

Mobile: +91-8978078449.





From: Kranthi Kumar K
Sent: Friday, January 18, 2019 4:22 PM
To: d...@lucene.apache.org; 
solr-user@lucene.apache.org
Cc: Ananda Babu medida 
mailto:anandababu.med...@ccubefintech.com>>;
 Srinivasa Reddy Karri 
mailto:srinivasareddy.ka...@ccubefintech.com>>;
 Michelle Ngo mailto:michelle@ccube.com.au>>; 
Ravi Vangala 
mailto:ravi.vang...@ccubefintech.com>>
Subject: RE: Solr Size Limitation upto 32 kb limitation



Hi team,



Thank you Erick Erickson ,Bernd Fehling , Jan Hoydahl for your suggested 
solutions. I’ve tried the suggested one’s and still we are unable to import 
files havingsize  >32 kb, it is displaying same error.



Below link has the suggested solutions. Please have a look once.



http://lucene.472066.n3.nabble.com/Solr-Size-Limitation-upto-32-KB-files-td4419779.html



  1.  As per Erick Erickson, I’ve changed the string type to Text type based 
and still the issue occurs .

I’ve changed from :







Changed to:







If we do so, it is showing error in the log, please find the error in the 
attachment.



If I change to:







It is not showing any error , but the issue still exists.



  1.  As per Jan Hoydahl, I have gone through the link that you have provided 
and checked ‘requestParsers’ tag in solrconfig.xml,



RequestParsers tag in our application is as follows:



‘’

Request parsers, which we are using and in the link you have provided are 
similar. And still we are unable to import the files size >32 kb.



  1.  As per Bernd Fehling, we are using Solr 4.10.2. you have mentioned as,

‘If you are trying to add larger content then you have to "chop" that
by yourself and add it as multivalued. Can be done within a self written 
loader. ’



I’m a newbie to Solr and I didn’t get what exactly ‘self written loader’ is?



Could you please provide us sample code, that helps us to go further?





[image001]

Thanks & Regards,

Kranthi Kumar.K,

Software Engineer,

Ccube Fintech Global Services Pvt Ltd.,

Email/Skype: 
kranthikuma...@ccubefintech.com,

Mobile: +91-8978078449.





From: Kranthi Kumar K 
mailto:kranthikuma...@ccubefintech.com>>
Sent: Thursday, January 17, 2019 12:43 PM
To: d...@lucene.apache.org; 
solr-user@lucene.apache.org
Cc: Ananda Babu medida 
mailto:anandababu.med...@ccubefintech.com>>;
 Srinivasa Reddy Karri 
mailto:srinivasareddy.ka...@ccubefintech.com>>;
 Michelle Ngo mailto:michelle@ccube.com.au>>
Subject: Re: Solr Size Limitation upto 32 kb limitation



Hi Team,



Can we have any updates on the below issue? We are awaiting your reply.



Thanks,

Kranthi kumar.K



From: Kranthi Kumar K
Sent: Friday, January 4, 2019 5:01:38 PM
To: d...@lucene.apache.org
Cc: Ananda Babu medida; Srinivasa Reddy Karri
Subject: Solr Size Limitation upto 32 kb limitation



Hi team,



We are currently using Solr 4.2.1 version in our project and everything is 
going well. But recently, we are facing an issue with Solr Data Import. It is 
not importing the files with size greater than 32766 bytes (i.e, 

Need to perfom search and group the record on basis of domain,subject,from address and display the count of label i.e inbox,spam

[swap]

Need to perfom search and group the record on basis of domain,subject,from
address and display the count of label i.e inbox,spam
  and label status i.e read and unread with it.The label and label status
should be displayed as percentage.

Scenorio 1
Document structure is as mentioned below indexed in solr. message_id is
unique field in solr
  {
"email_date_time": 1548922689,
"subject": "abcdef",
"created": 1548932108,
"domain": ".com",
"message_id": "123456789ui",
"label": "inbox",
"from_address": xxxbc.com",
"email": "g...@gmail.com",
"label_status": "unread"
  }

  {
"email_date_time": 1548922689,
"subject": "abcdef",
"created": 1548932108,
"domain": ".com",
"message_id": "zxiu22",
"label": "inbox",
"from_address": xxxbc.com",
"email": "g...@gmail.com",
"label_status": "unread"
  }

  {
"email_date_time": 1548922689,
"subject": "defg",
"created": 1548932108,
"domain": ".com",
"message_id": "ftyuiooo899",
"label": "inbox",
"from_address": xxxbc.com",
"email": "f...@gmail.com",
"label_status": "unread"
  }

I have below mentioned point to be implemented

1. Need to perfom search and group the record on basis of
domain,subject,from address and display the count of label i.e inbox,spam
  and label status i.e read and unread with it.The label and label status
should be displayed as percentage.

 
2. Need to paginate the record along with the implementation 1


Display will be as mentioned below


1. domain name : @ subject:hello from addredd: abcd@i

inbox percentage : 20% spam percentage : 80% 
read percentage  : 30%  unread percentage : 70%

2. domain name : @ subject:hi from addredd: abcd@i

inbox percentage : 20% spam percentage : 80% 
read percentage  : 30%  unread percentage : 70%


3. domain name : @ subject:where from addredd: abcd@i

inbox percentage : 20% spam percentage : 80% 
read percentage  : 30%  unread percentage : 70%



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Query over nested documents with an AND Operator

[Gelszus Julia]

Hello,
recently I started to work with Apache SOLR 7.6.0 in the course of a project. I 
encountered the following problem during the queries, where I want to know if 
the SOLR offers this in general:
We want to index a nested XML document, which can have the following structure:
- there are 1-n Parent Documents which each contain the article data (short 
text, long text etc.)
-in addition, each Parent Document can have 0-n Child Documents containing the 
attributes of the article
-example of an XML file:

  1
  4711
  here is a short text dealing with plastic and 
brass
  here is a detailed description
  parentDocument
  

  2
  4811
  here is a shorttext
  here you will find a detailed description
  parentDocument
  

  2_1
  material 
  brass
  
  
  2_2
  material quality
  plastic
  


I need an AND operator between my queries because I want to get as accurate 
hits as possible. I managed to search all Parent and Child Documents with one 
search term and get the right result.
But if I want to search for example for plastic and brass (that means 2 or more 
search terms). I want to get both the Parent Document for the respective child 
document as result (article 4811), as well as article 4711 because in this 
article the two words appear in the description. But the result of my query is 
always only article 4711. I know that I could also write the attribute in one 
field. However, I want to have a facet about the attribute name.

I hope you can help me with this problem.

Thank you very much,

Mit freundlichen Grüßen / Kind regards

Julia Gelszus

Bachelor of Science
Consultant SAP Development Workbench

FIS Informationssysteme und Consulting GmbH
Röthleiner Weg 1
97506 Grafenrheinfeld

P +49 (9723) 9188-667
F +49 (9723) 9188-200
E j.gels...@fis-gmbh.de
www.fis-gmbh.de

Managing Directors:
Ralf Bernhardt, Wolfgang Ebner, Frank Schöngarth

Registration Office Schweinfurt HRB 2209

[cid:image001.jpg@01D4B9FB.5FCEDF70]  
[cid:image002.jpg@01D4B9FB.5FCEDF70]    
[cid:image003.jpg@01D4B9FB.5FCEDF70] 
   
[cid:image004.jpg@01D4B9FB.5FCEDF70] 
   
[cid:image005.jpg@01D4B9FB.5FCEDF70] 

Re: Creating shard with core.properties

[Bharath Kumar]

Thanks Shawn for your inputs and the pointer to the documentation. Our
setup currently has 1 shard and 2 replicas for that shard and we do not
want a manual step which involves creating a collection since for SOLR
Cloud at least more than 50% of the shard nodes should be up and running.
Also if the zookeeper states go bad for some reason, we will need to
re-create the collection, whereas in the legacy cloud mode with manual
core.properties creation it has helped us bring up the solr cloud even
without any known zookeeper states after an upgrade and not do any
additional step.

On Wed, Jan 30, 2019 at 3:49 PM Shawn Heisey  wrote:

> On 1/30/2019 3:36 PM, Bharath Kumar wrote:
> > Thanks Erick. We cleanup the zookeeper state on every installation, so
> the
> > zookeeper states are gone. So what should we do in case of a new 7.6
> > installation where we want to manually create core.properties and use the
> > non-legacy cloud option? Is it in order to use non-legacy cloud, we
> should
> > use the collections api to create a collection first and then use the
> > manual core.properties for auto-discovery?
>
> *ALL* creations and modifications to SolrCloud collections should be
> done using the Collections API.  Creating cores directly (either with
> core.properties or the CoreAdmin API) is something that will almost
> certainly bite you hard.  Based on what Erick has said, I don't think
> you can even do it at all when legacy mode is disabled.  Even when you
> can ... don't.
>
> > Because in the legacy cloud mode we were just creating the
> core.properties
> > manually and that would update the zookeeper state when the solr boots
> up.
> > Can you please help me with this?
>
> Use the Collections API.  This is the recommendation even for experts
> who really know the code.  Creating cores manually in ANY SolrCloud
> install is a recipe for problems, even in legacy mode.
>
> There is a very large warning box (red triangle with an exclamation
> point) in this section of the documentation:
>
>
> https://lucene.apache.org/solr/guide/7_6/coreadmin-api.html#coreadmin-create
>
> One of the first things it says there in that warning box is that the
> CoreAdmin API should not be used in SolrCloud.  Manually creating
> core.properties files and restarting Solr is effectively the same thing
> as using the CoreAdmin API.
>
> Thanks,
> Shawn
>


-- 
Thanks & Regards,
Bharath MV Kumar

"Life is short, enjoy every moment of it"