Re: REINDEXCOLLECTION not working on an alias

[Bjarke Buur Mortensen]

OK, that makes sense.
 Looking forward to that fix, thanks for the reply.

Den tir. 19. maj 2020 kl. 17.21 skrev Joel Bernstein :

> I believe the issue is that under the covers this feature is using the
> "topic" streaming expressions which it was just reported doesn't work with
> aliases. This is something that will get fixed, but for the current release
> there isn't a workaround for this issue.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Tue, May 19, 2020 at 8:25 AM Bjarke Buur Mortensen <
> morten...@eluence.com>
> wrote:
>
> > Hi list,
> >
> > I seem to be unable to get REINDEXCOLLECTION to work on a collection
> alias
> > (running Solr 8.2.0). The documentation seems to state that that should
> be
> > possible:
> >
> >
> https://lucene.apache.org/solr/guide/8_2/collection-management.html#reindexcollection
> > "name
> > Source collection name, may be an alias. This parameter is required."
> >
> > If I run on my alias (qa_supplier_products):
> > curl "
> >
> >
> http://localhost:8983/solr/admin/collections?action=REINDEXCOLLECTION=qa_supplier_products=1=start
> > I get an error:
> > "org.apache.solr.common.SolrException: Unable to copy documents from
> > qa_supplier_products to .rx_qa_supplier_products_6:
> > {\"result-set\":{\"docs\":[\n
> >  {\"DaemonOp\":\"Deamon:.rx_qa_supplier_products_6 started on
> > .rx_qa_supplier_products_0_shard1_replica_n1\"
> >
> > If I instead point to the underlying collection, everything works fine.
> Now
> > I have an alias pointing to an alias, which works, but ideally I would
> like
> > to just have my main alias point to the newly reindexed collection.
> >
> > Can anybody help me out here?
> >
> > Thanks,
> > /Bjarke
> >
>

Re: REINDEXCOLLECTION not working on an alias

[Joel Bernstein]

I believe the issue is that under the covers this feature is using the
"topic" streaming expressions which it was just reported doesn't work with
aliases. This is something that will get fixed, but for the current release
there isn't a workaround for this issue.


Joel Bernstein
http://joelsolr.blogspot.com/


On Tue, May 19, 2020 at 8:25 AM Bjarke Buur Mortensen 
wrote:

> Hi list,
>
> I seem to be unable to get REINDEXCOLLECTION to work on a collection alias
> (running Solr 8.2.0). The documentation seems to state that that should be
> possible:
>
> https://lucene.apache.org/solr/guide/8_2/collection-management.html#reindexcollection
> "name
> Source collection name, may be an alias. This parameter is required."
>
> If I run on my alias (qa_supplier_products):
> curl "
>
> http://localhost:8983/solr/admin/collections?action=REINDEXCOLLECTION=qa_supplier_products=1=start
> I get an error:
> "org.apache.solr.common.SolrException: Unable to copy documents from
> qa_supplier_products to .rx_qa_supplier_products_6:
> {\"result-set\":{\"docs\":[\n
>  {\"DaemonOp\":\"Deamon:.rx_qa_supplier_products_6 started on
> .rx_qa_supplier_products_0_shard1_replica_n1\"
>
> If I instead point to the underlying collection, everything works fine. Now
> I have an alias pointing to an alias, which works, but ideally I would like
> to just have my main alias point to the newly reindexed collection.
>
> Can anybody help me out here?
>
> Thanks,
> /Bjarke
>

RE: highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi Edwin,

Thanks for your explanation, makes sense now.

Best regards

Martin


Internal - KMD A/S

-Original Message-
From: Zheng Lin Edwin Yeo 
Sent: 30. juni 2019 01:57
To: solr-user@lucene.apache.org
Subject: Re: highlighting not working as expected

Hi,

If you are using the type "string", it will require exact match, including 
space and upper/lower case.

You can use the type "text" for a start, but further down the road it will be 
good to have your own custom fieldType with your own tokenizer and filter.

Regards,
Edwin

On Tue, 25 Jun 2019 at 14:52, Martin Frank Hansen (MHQ)  wrote:

> Hi again,
>
> I have tested a bit and I was wondering if the highlighter requires a
> field to be of type "text"? Whenever I try highlighting on fields
> which are of type "string" nothing gets returned.
>
> Best regards
>
> Martin
>
>
> Internal - KMD A/S
>
> -Original Message-
> From: Jörn Franke 
> Sent: 11. juni 2019 08:45
> To: solr-user@lucene.apache.org
> Subject: Re: highlighting not working as expected
>
> Could it be a stop word ? What is the exact type definition of those
> fields? Could this word be omitted or with wrong encoding during
> loading of the documents?
>
> > Am 03.06.2019 um 10:06 schrieb Martin Frank Hansen (MHQ) :
> >
> > Hi,
> >
> > I am having some difficulties making highlighting work. For some
> > reason
> the highlighting feature only works on some fields but not on other
> fields even though these fields are stored.
> >
> > An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagst
> itel=Sagstitel=%3C/b%3E=%3Cb%3E=
> on=rotte
> >
> > It simply returns an empty set, for all documents even though I can
> > see
> several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
> >
> > I am using the standard highlighter as below.
> >
> >
> > 
> >
> >  
> >  
> >   >  default="true"
> >  class="solr.highlight.GapFragmenter">
> >
> >  100
> >
> >  
> >
> >  
> >   >  class="solr.highlight.RegexFragmenter">
> >
> >  
> >  70
> >  
> >  0.5
> >  
> >  [-\w
> ,/\n\]{20,200}
> >
> >  
> >
> >  
> >   > default="true"
> > class="solr.highlight.HtmlFormatter">
> >
> >  b
> >  /b
> >
> >  
> >
> >  
> >   >   class="solr.highlight.HtmlEncoder" />
> >
> >  
> >   >   class="solr.highlight.SimpleFragListBuilder"/>
> >
> >  
> >   >   class="solr.highlight.SingleFragListBuilder"/>
> >
> >  
> >  >   default="true"
> >
> > class="solr.highlight.WeightedFragListBuilder"/>
> >
> >  
> >   >default="true"
> >class="solr.highlight.ScoreOrderFragmentsBuilder">
> >
> >  
> >
> >  
> >   >class="solr.highlight.ScoreOrderFragmentsBuilder">
> >
> >  
> >  
> >
> >  
> >
> >   >   default="true"
> >   class="solr.highlight.SimpleBoundaryScanner">
> >
> >  10
> >  .,!? 
> >
> >  
> >
> >   >
>  class="solr.highlight.BreakIteratorBoundaryScanner">
> >
> >  
> >  WORD
> >  
> >  
> >  da
> >
> >  
> >
> >  
> >
> > Hope that some one can help, thanks in advance.
> >
> > Best regards
> > Martin
> >
> >
> >
> > Internal - KMD A/S
> >
> > Beskyttelse af dine personlige oplysninger er vigtig for os. Her
> > finder
> du KMD’s Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der
> fortæller, hvordan vi behandler oplysninger om dig.
> >
> > Protection of your personal dat

Re: highlighting not working as expected

[Zheng Lin Edwin Yeo]

Hi,

If you are using the type "string", it will require exact match, including
space and upper/lower case.

You can use the type "text" for a start, but further down the road it will
be good to have your own custom fieldType with your own tokenizer and
filter.

Regards,
Edwin

On Tue, 25 Jun 2019 at 14:52, Martin Frank Hansen (MHQ)  wrote:

> Hi again,
>
> I have tested a bit and I was wondering if the highlighter requires a
> field to be of type "text"? Whenever I try highlighting on fields which are
> of type "string" nothing gets returned.
>
> Best regards
>
> Martin
>
>
> Internal - KMD A/S
>
> -Original Message-
> From: Jörn Franke 
> Sent: 11. juni 2019 08:45
> To: solr-user@lucene.apache.org
> Subject: Re: highlighting not working as expected
>
> Could it be a stop word ? What is the exact type definition of those
> fields? Could this word be omitted or with wrong encoding during loading of
> the documents?
>
> > Am 03.06.2019 um 10:06 schrieb Martin Frank Hansen (MHQ) :
> >
> > Hi,
> >
> > I am having some difficulties making highlighting work. For some reason
> the highlighting feature only works on some fields but not on other fields
> even though these fields are stored.
> >
> > An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
> >
> > It simply returns an empty set, for all documents even though I can see
> several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
> >
> > I am using the standard highlighter as below.
> >
> >
> > 
> >
> >  
> >  
> >   >  default="true"
> >  class="solr.highlight.GapFragmenter">
> >
> >  100
> >
> >  
> >
> >  
> >   >  class="solr.highlight.RegexFragmenter">
> >
> >  
> >  70
> >  
> >  0.5
> >  
> >  [-\w
> ,/\n\]{20,200}
> >
> >  
> >
> >  
> >   > default="true"
> > class="solr.highlight.HtmlFormatter">
> >
> >  b
> >  /b
> >
> >  
> >
> >  
> >   >   class="solr.highlight.HtmlEncoder" />
> >
> >  
> >   >   class="solr.highlight.SimpleFragListBuilder"/>
> >
> >  
> >   >   class="solr.highlight.SingleFragListBuilder"/>
> >
> >  
> >  >   default="true"
> >   class="solr.highlight.WeightedFragListBuilder"/>
> >
> >  
> >   >default="true"
> >class="solr.highlight.ScoreOrderFragmentsBuilder">
> >
> >  
> >
> >  
> >   >class="solr.highlight.ScoreOrderFragmentsBuilder">
> >
> >  
> >  
> >
> >  
> >
> >   >   default="true"
> >   class="solr.highlight.SimpleBoundaryScanner">
> >
> >  10
> >  .,!? 
> >
> >  
> >
> >   >
>  class="solr.highlight.BreakIteratorBoundaryScanner">
> >
> >  
> >  WORD
> >  
> >  
> >  da
> >
> >  
> >
> >  
> >
> > Hope that some one can help, thanks in advance.
> >
> > Best regards
> > Martin
> >
> >
> >
> > Internal - KMD A/S
> >
> > Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder
> du KMD’s Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der
> fortæller, hvordan vi behandler oplysninger om dig.
> >
> > Protection of your personal data is important to us. Here you can read
> KMD’s Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how we
> process your personal data.
> >
> > Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig
> information. Hvis du ved en fejltagelse modtager e-mail

RE: highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi again,

I have tested a bit and I was wondering if the highlighter requires a field to 
be of type "text"? Whenever I try highlighting on fields which are of type 
"string" nothing gets returned.

Best regards

Martin


Internal - KMD A/S

-Original Message-
From: Jörn Franke 
Sent: 11. juni 2019 08:45
To: solr-user@lucene.apache.org
Subject: Re: highlighting not working as expected

Could it be a stop word ? What is the exact type definition of those fields? 
Could this word be omitted or with wrong encoding during loading of the 
documents?

> Am 03.06.2019 um 10:06 schrieb Martin Frank Hansen (MHQ) :
>
> Hi,
>
> I am having some difficulties making highlighting work. For some reason the 
> highlighting feature only works on some fields but not on other fields even 
> though these fields are stored.
>
> An example of a request looks like this: 
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
>
> It simply returns an empty set, for all documents even though I can see 
> several documents which have “Sagstitel” containing the word “rotte” 
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
>
>  
>  
>default="true"
>  class="solr.highlight.GapFragmenter">
>
>  100
>
>  
>
>  
>class="solr.highlight.RegexFragmenter">
>
>  
>  70
>  
>  0.5
>  
>  [-\w ,/\n\]{20,200}
>
>  
>
>  
>   default="true"
> class="solr.highlight.HtmlFormatter">
>
>  b
>  /b
>
>  
>
>  
> class="solr.highlight.HtmlEncoder" />
>
>  
> class="solr.highlight.SimpleFragListBuilder"/>
>
>  
> class="solr.highlight.SingleFragListBuilder"/>
>
>  
>default="true"
>   class="solr.highlight.WeightedFragListBuilder"/>
>
>  
>  default="true"
>class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
>
>  
>  class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
>  
>
>  
>
> default="true"
>   class="solr.highlight.SimpleBoundaryScanner">
>
>  10
>  .,!? 
>
>  
>
> class="solr.highlight.BreakIteratorBoundaryScanner">
>
>  
>  WORD
>  
>  
>  da
>
>  
>
>  
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du 
> KMD’s Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der fortæller, 
> hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read KMD’s 
> Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how we process 
> your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. 
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere 
> afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette 
> e-mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen 
> og ethvert vedhæftet bilag efter vores overbevisning er fri for virus og 
> andre fejl, som kan påvirke computeren eller it-systemet, hvori den modtages 
> og læses, åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget 
> ansvar for tab og skade, som er opstået i forbindelse med at modtage og bruge 
> e-mailen.
>
> Please note that this message may contain confidential information. If you 
> have received this message by mistake, please inform the sender of the 
> mistake by sending a reply, then delete the message from your system without 
> making, distributing or retaining any copies of it. Although we believe that 
> the message and any attachments are free from viruses and other errors that 
> might affect the computer or it-system where it is received and read, the 
> recipient opens the message at his or her own risk. We assume no 
> responsibility for any loss or damage arising from the receipt or use of this 
> message.

RE: highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi Edwin,

Yes the field is defined just like the other fields:



BR
Martin


Internal - KMD A/S

-Original Message-
From: Zheng Lin Edwin Yeo 
Sent: 4. juni 2019 10:32
To: solr-user@lucene.apache.org
Subject: Re: highlighting not working as expected

Hi Martin,

What fieldType are you using for the field “Sagstitel”? Is it the same as other 
fields?

Regards,
Edwin

On Mon, 3 Jun 2019 at 16:06, Martin Frank Hansen (MHQ)  wrote:

> Hi,
>
> I am having some difficulties making highlighting work. For some
> reason the highlighting feature only works on some fields but not on
> other fields even though these fields are stored.
>
> An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagst
> itel=Sagstitel=%3C/b%3E=%3Cb%3E=
> on=rotte
>
> It simply returns an empty set, for all documents even though I can
> see several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
> 
>   
>   
>  default="true"
>   class="solr.highlight.GapFragmenter">
> 
>   100
> 
>   
>
>   
>  class="solr.highlight.RegexFragmenter">
> 
>   
>   70
>   
>   0.5
>   
>   [-\w
> ,/\n\]{20,200}
> 
>   
>
>   
> default="true"
>  class="solr.highlight.HtmlFormatter">
> 
>   b
>   /b
> 
>   
>
>   
>   class="solr.highlight.HtmlEncoder" />
>
>   
>   class="solr.highlight.SimpleFragListBuilder"/>
>
>   
>   class="solr.highlight.SingleFragListBuilder"/>
>
>   
>  default="true"
>
> class="solr.highlight.WeightedFragListBuilder"/>
>
>   
>default="true"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>
>   
>class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>   
> 
>   
>
>   default="true"
>class="solr.highlight.SimpleBoundaryScanner">
> 
>   10
>   .,!? 
> 
>   
>
>   class="solr.highlight.BreakIteratorBoundaryScanner">
> 
>   
>   WORD
>   
>   
>   da
> 
>   
> 
>   
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her
> finder du KMD’s
> Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der fortæller, 
> hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read
> KMD’s Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how
> we process your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information.
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst
> informere afsender om fejlen ved at bruge svarfunktionen. Samtidig
> beder vi dig slette e-mailen i dit system uden at videresende eller kopiere 
> den.
> Selvom e-mailen og ethvert vedhæftet bilag efter vores overbevisning
> er fri for virus og andre fejl, som kan påvirke computeren eller
> it-systemet, hvori den modtages og læses, åbnes den på modtagerens
> eget ansvar. Vi påtager os ikke noget ansvar for tab og skade, som er
> opstået i forbindelse med at modtage og bruge e-mailen.
>
> Please note that this message may contain confidential information. If
> you have received this message by mistake, please inform the sender of
> the mistake by sending a reply, then delete the message from your
> system without making, distributing or retaining any copies of it.
> Although we believe that the message and any attachments are free from
> viruses and other errors that might affect the computer or it-system
> where it is received and read, the recipient opens the message at his or her 
> own risk.
> We assume no responsibility for any loss or damage arising from the
> receipt or use of this message.
>

RE: highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi Jörn,

Thanks for your input!

I do not use stop-words, so that should not be the issue. The encoding of the 
documents might be an issue, as they come in many different file formats. It 
will however need to test this.

The field is defined as below:



BR

Martin


Internal - KMD A/S

-Original Message-
From: Jörn Franke 
Sent: 11. juni 2019 08:45
To: solr-user@lucene.apache.org
Subject: Re: highlighting not working as expected

Could it be a stop word ? What is the exact type definition of those fields? 
Could this word be omitted or with wrong encoding during loading of the 
documents?

> Am 03.06.2019 um 10:06 schrieb Martin Frank Hansen (MHQ) :
>
> Hi,
>
> I am having some difficulties making highlighting work. For some reason the 
> highlighting feature only works on some fields but not on other fields even 
> though these fields are stored.
>
> An example of a request looks like this: 
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
>
> It simply returns an empty set, for all documents even though I can see 
> several documents which have “Sagstitel” containing the word “rotte” 
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
>
>  
>  
>default="true"
>  class="solr.highlight.GapFragmenter">
>
>  100
>
>  
>
>  
>class="solr.highlight.RegexFragmenter">
>
>  
>  70
>  
>  0.5
>  
>  [-\w ,/\n\]{20,200}
>
>  
>
>  
>   default="true"
> class="solr.highlight.HtmlFormatter">
>
>  b
>  /b
>
>  
>
>  
> class="solr.highlight.HtmlEncoder" />
>
>  
> class="solr.highlight.SimpleFragListBuilder"/>
>
>  
> class="solr.highlight.SingleFragListBuilder"/>
>
>  
>default="true"
>   class="solr.highlight.WeightedFragListBuilder"/>
>
>  
>  default="true"
>class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
>
>  
>  class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
>  
>
>  
>
> default="true"
>   class="solr.highlight.SimpleBoundaryScanner">
>
>  10
>  .,!? 
>
>  
>
> class="solr.highlight.BreakIteratorBoundaryScanner">
>
>  
>  WORD
>  
>  
>  da
>
>  
>
>  
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du 
> KMD’s Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der fortæller, 
> hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read KMD’s 
> Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how we process 
> your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. 
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere 
> afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette 
> e-mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen 
> og ethvert vedhæftet bilag efter vores overbevisning er fri for virus og 
> andre fejl, som kan påvirke computeren eller it-systemet, hvori den modtages 
> og læses, åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget 
> ansvar for tab og skade, som er opstået i forbindelse med at modtage og bruge 
> e-mailen.
>
> Please note that this message may contain confidential information. If you 
> have received this message by mistake, please inform the sender of the 
> mistake by sending a reply, then delete the message from your system without 
> making, distributing or retaining any copies of it. Although we believe that 
> the message and any attachments are free from viruses and other errors that 
> might affect the computer or it-system where it is received and read, the 
> recipient opens the message at his or her own risk. We assume no 
> responsibility for any loss or damage arising from the receipt or use of this 
> message.

Re: highlighting not working as expected

[Jörn Franke]

Could it be a stop word ? What is the exact type definition of those fields? 
Could this word be omitted or with wrong encoding during loading of the 
documents?

> Am 03.06.2019 um 10:06 schrieb Martin Frank Hansen (MHQ) :
> 
> Hi,
> 
> I am having some difficulties making highlighting work. For some reason the 
> highlighting feature only works on some fields but not on other fields even 
> though these fields are stored.
> 
> An example of a request looks like this: 
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
> 
> It simply returns an empty set, for all documents even though I can see 
> several documents which have “Sagstitel” containing the word “rotte” 
> (rotte=rat).  What am I missing here?
> 
> I am using the standard highlighter as below.
> 
> 
> 
>
>  
>  
>default="true"
>  class="solr.highlight.GapFragmenter">
>
>  100
>
>  
> 
>  
>class="solr.highlight.RegexFragmenter">
>
>  
>  70
>  
>  0.5
>  
>  [-\w ,/\n\]{20,200}
>
>  
> 
>  
>   default="true"
> class="solr.highlight.HtmlFormatter">
>
>  b
>  /b
>
>  
> 
>  
> class="solr.highlight.HtmlEncoder" />
> 
>  
> class="solr.highlight.SimpleFragListBuilder"/>
> 
>  
> class="solr.highlight.SingleFragListBuilder"/>
> 
>  
>default="true"
>   class="solr.highlight.WeightedFragListBuilder"/>
> 
>  
>  default="true"
>class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
> 
>  
>  class="solr.highlight.ScoreOrderFragmentsBuilder">
>
>  
>  
>
>  
> 
> default="true"
>   class="solr.highlight.SimpleBoundaryScanner">
>
>  10
>  .,!? 
>
>  
> 
> class="solr.highlight.BreakIteratorBoundaryScanner">
>
>  
>  WORD
>  
>  
>  da
>
>  
>
>  
> 
> Hope that some one can help, thanks in advance.
> 
> Best regards
> Martin
> 
> 
> 
> Internal - KMD A/S
> 
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du 
> KMD’s Privatlivspolitik, der fortæller, 
> hvordan vi behandler oplysninger om dig.
> 
> Protection of your personal data is important to us. Here you can read KMD’s 
> Privacy Policy outlining how we process 
> your personal data.
> 
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. 
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere 
> afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette 
> e-mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen 
> og ethvert vedhæftet bilag efter vores overbevisning er fri for virus og 
> andre fejl, som kan påvirke computeren eller it-systemet, hvori den modtages 
> og læses, åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget 
> ansvar for tab og skade, som er opstået i forbindelse med at modtage og bruge 
> e-mailen.
> 
> Please note that this message may contain confidential information. If you 
> have received this message by mistake, please inform the sender of the 
> mistake by sending a reply, then delete the message from your system without 
> making, distributing or retaining any copies of it. Although we believe that 
> the message and any attachments are free from viruses and other errors that 
> might affect the computer or it-system where it is received and read, the 
> recipient opens the message at his or her own risk. We assume no 
> responsibility for any loss or damage arising from the receipt or use of this 
> message.

RE: highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi David,

Thanks for your response and sorry my late reply.

Still the same result when using hl.method=unified.

Best regards
Martin


Internal - KMD A/S

-Original Message-
From: David Smiley 
Sent: 10. juni 2019 16:48
To: solr-user 
Subject: Re: highlighting not working as expected

Please try hl.method=unified and tell us if that helps.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Jun 3, 2019 at 4:06 AM Martin Frank Hansen (MHQ)  wrote:

> Hi,
>
> I am having some difficulties making highlighting work. For some
> reason the highlighting feature only works on some fields but not on
> other fields even though these fields are stored.
>
> An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagst
> itel=Sagstitel=%3C/b%3E=%3Cb%3E=
> on=rotte
>
> It simply returns an empty set, for all documents even though I can
> see several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
> 
>   
>   
>  default="true"
>   class="solr.highlight.GapFragmenter">
> 
>   100
> 
>   
>
>   
>  class="solr.highlight.RegexFragmenter">
> 
>   
>   70
>   
>   0.5
>   
>   [-\w
> ,/\n\]{20,200}
> 
>   
>
>   
> default="true"
>  class="solr.highlight.HtmlFormatter">
> 
>   b
>   /b
> 
>   
>
>   
>   class="solr.highlight.HtmlEncoder" />
>
>   
>   class="solr.highlight.SimpleFragListBuilder"/>
>
>   
>   class="solr.highlight.SingleFragListBuilder"/>
>
>   
>  default="true"
>
> class="solr.highlight.WeightedFragListBuilder"/>
>
>   
>default="true"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>
>   
>class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>   
> 
>   
>
>   default="true"
>class="solr.highlight.SimpleBoundaryScanner">
> 
>   10
>   .,!? 
> 
>   
>
>   class="solr.highlight.BreakIteratorBoundaryScanner">
> 
>   
>   WORD
>   
>   
>   da
> 
>   
> 
>   
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her
> finder du KMD’s
> Privatlivspolitik<http://www.kmd.dk/Privatlivspolitik>, der fortæller, 
> hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read
> KMD’s Privacy Policy<http://www.kmd.net/Privacy-Policy> outlining how
> we process your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information.
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst
> informere afsender om fejlen ved at bruge svarfunktionen. Samtidig
> beder vi dig slette e-mailen i dit system uden at videresende eller kopiere 
> den.
> Selvom e-mailen og ethvert vedhæftet bilag efter vores overbevisning
> er fri for virus og andre fejl, som kan påvirke computeren eller
> it-systemet, hvori den modtages og læses, åbnes den på modtagerens
> eget ansvar. Vi påtager os ikke noget ansvar for tab og skade, som er
> opstået i forbindelse med at modtage og bruge e-mailen.
>
> Please note that this message may contain confidential information. If
> you have received this message by mistake, please inform the sender of
> the mistake by sending a reply, then delete the message from your
> system without making, distributing or retaining any copies of it.
> Although we believe that the message and any attachments are free from
> viruses and other errors that might affect the computer or it-system
> where it is received and read, the recipient opens the message at his or her 
> own risk.
> We assume no responsibility for any loss or damage arising from the
> receipt or use of this message.
>

Re: highlighting not working as expected

[David Smiley]

Please try hl.method=unified and tell us if that helps.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Mon, Jun 3, 2019 at 4:06 AM Martin Frank Hansen (MHQ)  wrote:

> Hi,
>
> I am having some difficulties making highlighting work. For some reason
> the highlighting feature only works on some fields but not on other fields
> even though these fields are stored.
>
> An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
>
> It simply returns an empty set, for all documents even though I can see
> several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
> 
>   
>   
>  default="true"
>   class="solr.highlight.GapFragmenter">
> 
>   100
> 
>   
>
>   
>  class="solr.highlight.RegexFragmenter">
> 
>   
>   70
>   
>   0.5
>   
>   [-\w
> ,/\n\]{20,200}
> 
>   
>
>   
> default="true"
>  class="solr.highlight.HtmlFormatter">
> 
>   b
>   /b
> 
>   
>
>   
>   class="solr.highlight.HtmlEncoder" />
>
>   
>   class="solr.highlight.SimpleFragListBuilder"/>
>
>   
>   class="solr.highlight.SingleFragListBuilder"/>
>
>   
>  default="true"
>class="solr.highlight.WeightedFragListBuilder"/>
>
>   
>default="true"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>
>   
>class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>   
> 
>   
>
>   default="true"
>class="solr.highlight.SimpleBoundaryScanner">
> 
>   10
>   .,!? 
> 
>   
>
>   class="solr.highlight.BreakIteratorBoundaryScanner">
> 
>   
>   WORD
>   
>   
>   da
> 
>   
> 
>   
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du
> KMD’s Privatlivspolitik, der
> fortæller, hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read
> KMD’s Privacy Policy outlining how we
> process your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information.
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst
> informere afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi
> dig slette e-mailen i dit system uden at videresende eller kopiere den.
> Selvom e-mailen og ethvert vedhæftet bilag efter vores overbevisning er fri
> for virus og andre fejl, som kan påvirke computeren eller it-systemet,
> hvori den modtages og læses, åbnes den på modtagerens eget ansvar. Vi
> påtager os ikke noget ansvar for tab og skade, som er opstået i forbindelse
> med at modtage og bruge e-mailen.
>
> Please note that this message may contain confidential information. If you
> have received this message by mistake, please inform the sender of the
> mistake by sending a reply, then delete the message from your system
> without making, distributing or retaining any copies of it. Although we
> believe that the message and any attachments are free from viruses and
> other errors that might affect the computer or it-system where it is
> received and read, the recipient opens the message at his or her own risk.
> We assume no responsibility for any loss or damage arising from the receipt
> or use of this message.
>

Re: highlighting not working as expected

[Zheng Lin Edwin Yeo]

Hi Martin,

What fieldType are you using for the field “Sagstitel”? Is it the same as
other fields?

Regards,
Edwin

On Mon, 3 Jun 2019 at 16:06, Martin Frank Hansen (MHQ)  wrote:

> Hi,
>
> I am having some difficulties making highlighting work. For some reason
> the highlighting feature only works on some fields but not on other fields
> even though these fields are stored.
>
> An example of a request looks like this:
> http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel=Sagstitel=%3C/b%3E=%3Cb%3E=on=rotte
>
> It simply returns an empty set, for all documents even though I can see
> several documents which have “Sagstitel” containing the word “rotte”
> (rotte=rat).  What am I missing here?
>
> I am using the standard highlighter as below.
>
>
> 
> 
>   
>   
>  default="true"
>   class="solr.highlight.GapFragmenter">
> 
>   100
> 
>   
>
>   
>  class="solr.highlight.RegexFragmenter">
> 
>   
>   70
>   
>   0.5
>   
>   [-\w
> ,/\n\]{20,200}
> 
>   
>
>   
> default="true"
>  class="solr.highlight.HtmlFormatter">
> 
>   b
>   /b
> 
>   
>
>   
>   class="solr.highlight.HtmlEncoder" />
>
>   
>   class="solr.highlight.SimpleFragListBuilder"/>
>
>   
>   class="solr.highlight.SingleFragListBuilder"/>
>
>   
>  default="true"
>class="solr.highlight.WeightedFragListBuilder"/>
>
>   
>default="true"
> class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>
>   
>class="solr.highlight.ScoreOrderFragmentsBuilder">
> 
>   
>   
> 
>   
>
>   default="true"
>class="solr.highlight.SimpleBoundaryScanner">
> 
>   10
>   .,!? 
> 
>   
>
>   class="solr.highlight.BreakIteratorBoundaryScanner">
> 
>   
>   WORD
>   
>   
>   da
> 
>   
> 
>   
>
> Hope that some one can help, thanks in advance.
>
> Best regards
> Martin
>
>
>
> Internal - KMD A/S
>
> Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du
> KMD’s Privatlivspolitik, der
> fortæller, hvordan vi behandler oplysninger om dig.
>
> Protection of your personal data is important to us. Here you can read
> KMD’s Privacy Policy outlining how we
> process your personal data.
>
> Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information.
> Hvis du ved en fejltagelse modtager e-mailen, beder vi dig venligst
> informere afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi
> dig slette e-mailen i dit system uden at videresende eller kopiere den.
> Selvom e-mailen og ethvert vedhæftet bilag efter vores overbevisning er fri
> for virus og andre fejl, som kan påvirke computeren eller it-systemet,
> hvori den modtages og læses, åbnes den på modtagerens eget ansvar. Vi
> påtager os ikke noget ansvar for tab og skade, som er opstået i forbindelse
> med at modtage og bruge e-mailen.
>
> Please note that this message may contain confidential information. If you
> have received this message by mistake, please inform the sender of the
> mistake by sending a reply, then delete the message from your system
> without making, distributing or retaining any copies of it. Although we
> believe that the message and any attachments are free from viruses and
> other errors that might affect the computer or it-system where it is
> received and read, the recipient opens the message at his or her own risk.
> We assume no responsibility for any loss or damage arising from the receipt
> or use of this message.
>

Re: Backup not working in SOlr 6.6

[Shawn Heisey]

On 5/31/2019 10:57 AM, Chuck Reynolds wrote:

Hey guys I’m try to do a backup of my Solr cloud cluster but it is never 
starting.

When I execute the async backup command it returns quickly like I would expect 
with the following response



0111234



But the backup never starts.


My reply is a total shot in the dark.  It might turn out that SolrCloud 
doesn't work the way I am thinking, making what I'm about to say 
worthless.  If that's the case, I hope somebody who has intimate 
knowledge of the backup code can smack me and let me know I'm giving bad 
info.


I wonder if you're in a situation where the overseer is stuck on one of 
the messages in its queue and never gets around to even noticing your 
backup request.


If I am right, then the way I would fix it would be to clear the 
overseer queue, which lives in in zookeper at /overseer/queue ... and 
restart all the Solr nodes.  Then try the backup again.


You could look at your overseer queue in ZK and see whether it has items 
stacked up (other than the backup request) that are not clearing out. 
That's easy to do within the admin UI -- go to the Cloud tab and click 
on Tree.  Then open the overseer node and the queue node under that.  A 
healthy and responsive system would not have many items in the queue.  I 
can't tell you what's normal ... I don't have a live system running that 
I can look at.  After starting the cloud example, it has no entries 
under queue -- the node can't even be expanded.


Are there any error messages in the logs of any of your servers before 
your attempt at a backup?  My first thought would be to look for 
something related to the overseer.


Thanks,
Shawn

Re: autoGeneratePhraseQueries not working

[Alexandre Rafalovitch]

Ah oops. Did not realize the original text was missing spaces. Looked
like so many questions that did, I did not recheck the search query.

Go with Erick's explanation for this specific case. And keep my in
mind for input with spaces.

Regards,
   Alex.

On Tue, 16 Apr 2019 at 17:48, Erick Erickson  wrote:
>
> The issue isn’t SoW. What’s happening here is that the query _parser_ passes 
> my25word through as a single token, then WordDelimiterGraphFilterFactory 
> splits it up on number/letter changes after SoW is out of the picture. The 
> admin/analysis page will show you how this works.
>
> By fiddling with the settings in WordDelimiterGraphFilterFactory, you can get 
> close to auto phrase queries, in particular catenateall. But it’s not quite 
> the same thing under any circumstances as phrases.
>
> Best,
> Erick
>
> > On Apr 16, 2019, at 4:31 AM, Leonardo Francalanci 
> >  wrote:
> >
> > Thank you for the reply.
> > I'm using eDisMax, does it use the same parser as the Standard Query Parser 
> > then?
> > I think this behavior should be documented somehow... it's very confusing 
> > and to be honest I don't even remember how I got to the sow parameter... 
> > and I'm not sure what that means for all other queries I have
> >
> >Il martedì 16 aprile 2019, 13:09:26 CEST, Alexandre Rafalovitch 
> >  ha scritto:
> >
> > The issue is that the Standard Query Parser does pre-processing of the
> > query and splits it on whitespace beforehand (to deal with all the
> > special syntax). So, if you don't use quoted phrases then by the time
> > the field specific query analyzer chain kicks in, the text is already
> > pre-split and the analyzer only sees one (pre space-separated) token
> > at a time. So, the autoGeneratePhraseQueries does not work then. If
> > you use different parsers that send whole text in (e.g. FieldQParser),
> > then - I think - it will work.
> >
> > Or, like you discovered, sow=true tells the Standard Query Parser to
> > send it all together as well.
> >
> > It is a bit of a messy part of Solr, because the Admin Analysis page
> > sends the text to the query analyzer without splitting (it does not
> > use any Query Parser). So, that adds to the confusion.
> >
> > Regards,
> >   Alex.
> >
> > On Tue, 16 Apr 2019 at 10:53, Leonardo Francalanci
> >  wrote:
> >>
> >>   To add some information: using "sow=true" it seems to work.But I don't 
> >> understand why with "sow=false" it wouldn't work (can't find anything in 
> >> the docs about sow interaction with autoGeneratePhraseQueries); and the 
> >> implication of setting saw=true.
> >> I've found this:[SOLR-9185] Solr's edismax and "Lucene"/standard query 
> >> parsers should optionally not split on whitespace before sending terms to 
> >> analysis - ASF JIRA
> >>
> >> |
> >> |
> >> |  |
> >> [SOLR-9185] Solr's edismax and "Lucene"/standard query parsers should op...
> >>
> >>
> >>   |
> >>
> >>   |
> >>
> >>   |
> >>
> >>
> >> But it's very low level and I can't find any doc more "user friendly"
> >>
> >> Il martedì 16 aprile 2019, 09:00:08 CEST, Leonardo Francalanci 
> >>  ha scritto:
> >>
> >>   Hi,
> >>
> >> I'm using Solr 8.0.0  I can't get autoGeneratePhraseQueries to work (also 
> >> tried with 7.7.1 and same result):
> >>
> >> debug":{
> >> "rawquerystring":"TROUBLESHOOT:my25word",
> >> "querystring":"TROUBLESHOOT:my25word",
> >> "parsedquery":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
> >> "parsedquery_toString":"TROUBLESHOOT:my TROUBLESHOOT:25 
> >> TROUBLESHOOT:word",
> >>
> >> I expected something like
> >>
> >> "parsedquery":"TROUBLESHOOT:"my 25 word"
> >> Why isn't autoGeneratePhraseQueries generating a quoted string argument 
> >> when I query???
> >>
> >>
> >> This is my configuration:
> >>
> >>>> indexed="true"  stored="true"/>
> >>  >> positionIncrementGap="100" autoGeneratePhraseQueries="true">
> >>   
> >> 
> >> 
> >> 
> >>  >> ignoreCase="true"
> >> words="lang/stopwords_en.txt"
> >> />
> >>  >> generateWordParts="1" generateNumberParts="1" catenateWords="1" 
> >> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
> >> 
> >>  >> protected="protwords.txt"/>
> >> 
> >> 
> >>   
> >>   
> >> 
> >>  >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
> >>  >> ignoreCase="true"
> >> words="lang/stopwords_en.txt"
> >> />
> >>  >> generateWordParts="1" generateNumberParts="1" catenateWords="0" 
> >> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
> >> 
> >>  >> protected="protwords.txt"/>
> >> 
> >>   
> >> 
> >>  >> stored="true" multiValued="true" omitNorms="true"/>
> >>
>

Re: autoGeneratePhraseQueries not working

[Erick Erickson]

The issue isn’t SoW. What’s happening here is that the query _parser_ passes 
my25word through as a single token, then WordDelimiterGraphFilterFactory splits 
it up on number/letter changes after SoW is out of the picture. The 
admin/analysis page will show you how this works.

By fiddling with the settings in WordDelimiterGraphFilterFactory, you can get 
close to auto phrase queries, in particular catenateall. But it’s not quite the 
same thing under any circumstances as phrases.

Best,
Erick

> On Apr 16, 2019, at 4:31 AM, Leonardo Francalanci 
>  wrote:
> 
> Thank you for the reply.
> I'm using eDisMax, does it use the same parser as the Standard Query Parser 
> then?
> I think this behavior should be documented somehow... it's very confusing and 
> to be honest I don't even remember how I got to the sow parameter... and I'm 
> not sure what that means for all other queries I have
> 
>Il martedì 16 aprile 2019, 13:09:26 CEST, Alexandre Rafalovitch 
>  ha scritto:  
> 
> The issue is that the Standard Query Parser does pre-processing of the
> query and splits it on whitespace beforehand (to deal with all the
> special syntax). So, if you don't use quoted phrases then by the time
> the field specific query analyzer chain kicks in, the text is already
> pre-split and the analyzer only sees one (pre space-separated) token
> at a time. So, the autoGeneratePhraseQueries does not work then. If
> you use different parsers that send whole text in (e.g. FieldQParser),
> then - I think - it will work.
> 
> Or, like you discovered, sow=true tells the Standard Query Parser to
> send it all together as well.
> 
> It is a bit of a messy part of Solr, because the Admin Analysis page
> sends the text to the query analyzer without splitting (it does not
> use any Query Parser). So, that adds to the confusion.
> 
> Regards,
>   Alex.
> 
> On Tue, 16 Apr 2019 at 10:53, Leonardo Francalanci
>  wrote:
>> 
>>   To add some information: using "sow=true" it seems to work.But I don't 
>> understand why with "sow=false" it wouldn't work (can't find anything in the 
>> docs about sow interaction with autoGeneratePhraseQueries); and the 
>> implication of setting saw=true.
>> I've found this:[SOLR-9185] Solr's edismax and "Lucene"/standard query 
>> parsers should optionally not split on whitespace before sending terms to 
>> analysis - ASF JIRA
>> 
>> |
>> |
>> |  |
>> [SOLR-9185] Solr's edismax and "Lucene"/standard query parsers should op...
>> 
>> 
>>   |
>> 
>>   |
>> 
>>   |
>> 
>> 
>> But it's very low level and I can't find any doc more "user friendly"
>> 
>> Il martedì 16 aprile 2019, 09:00:08 CEST, Leonardo Francalanci 
>>  ha scritto:
>> 
>>   Hi,
>> 
>> I'm using Solr 8.0.0  I can't get autoGeneratePhraseQueries to work (also 
>> tried with 7.7.1 and same result):
>> 
>> debug":{
>> "rawquerystring":"TROUBLESHOOT:my25word",
>> "querystring":"TROUBLESHOOT:my25word",
>> "parsedquery":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
>> "parsedquery_toString":"TROUBLESHOOT:my TROUBLESHOOT:25 
>> TROUBLESHOOT:word",
>> 
>> I expected something like
>> 
>> "parsedquery":"TROUBLESHOOT:"my 25 word"
>> Why isn't autoGeneratePhraseQueries generating a quoted string argument when 
>> I query???
>> 
>> 
>> This is my configuration:
>> 
>>   > indexed="true"  stored="true"/>
>> > positionIncrementGap="100" autoGeneratePhraseQueries="true">
>>   
>> 
>> 
>> 
>> > ignoreCase="true"
>> words="lang/stopwords_en.txt"
>> />
>> > generateWordParts="1" generateNumberParts="1" catenateWords="1" 
>> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>> 
>> > protected="protwords.txt"/>
>> 
>> 
>>   
>>   
>> 
>> > synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
>> > ignoreCase="true"
>> words="lang/stopwords_en.txt"
>> />
>> > generateWordParts="1" generateNumberParts="1" catenateWords="0" 
>> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
>> 
>> > protected="protwords.txt"/>
>> 
>>   
>> 
>> > stored="true" multiValued="true" omitNorms="true"/>
>> 

Re: autoGeneratePhraseQueries not working

[Leonardo Francalanci]

 Thank you for the reply.
I'm using eDisMax, does it use the same parser as the Standard Query Parser 
then?
I think this behavior should be documented somehow... it's very confusing and 
to be honest I don't even remember how I got to the sow parameter... and I'm 
not sure what that means for all other queries I have

Il martedì 16 aprile 2019, 13:09:26 CEST, Alexandre Rafalovitch 
 ha scritto:  
 
 The issue is that the Standard Query Parser does pre-processing of the
query and splits it on whitespace beforehand (to deal with all the
special syntax). So, if you don't use quoted phrases then by the time
the field specific query analyzer chain kicks in, the text is already
pre-split and the analyzer only sees one (pre space-separated) token
at a time. So, the autoGeneratePhraseQueries does not work then. If
you use different parsers that send whole text in (e.g. FieldQParser),
then - I think - it will work.

Or, like you discovered, sow=true tells the Standard Query Parser to
send it all together as well.

It is a bit of a messy part of Solr, because the Admin Analysis page
sends the text to the query analyzer without splitting (it does not
use any Query Parser). So, that adds to the confusion.

Regards,
  Alex.

On Tue, 16 Apr 2019 at 10:53, Leonardo Francalanci
 wrote:
>
>  To add some information: using "sow=true" it seems to work.But I don't 
>understand why with "sow=false" it wouldn't work (can't find anything in the 
>docs about sow interaction with autoGeneratePhraseQueries); and the 
>implication of setting saw=true.
> I've found this:[SOLR-9185] Solr's edismax and "Lucene"/standard query 
> parsers should optionally not split on whitespace before sending terms to 
> analysis - ASF JIRA
>
> |
> |
> |  |
> [SOLR-9185] Solr's edismax and "Lucene"/standard query parsers should op...
>
>
>  |
>
>  |
>
>  |
>
>
> But it's very low level and I can't find any doc more "user friendly"
>
>    Il martedì 16 aprile 2019, 09:00:08 CEST, Leonardo Francalanci 
> ha scritto:
>
>  Hi,
>
> I'm using Solr 8.0.0  I can't get autoGeneratePhraseQueries to work (also 
> tried with 7.7.1 and same result):
>
> debug":{
>    "rawquerystring":"TROUBLESHOOT:my25word",
>    "querystring":"TROUBLESHOOT:my25word",
>    "parsedquery":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
>    "parsedquery_toString":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
>
> I expected something like
>
> "parsedquery":"TROUBLESHOOT:"my 25 word"
> Why isn't autoGeneratePhraseQueries generating a quoted string argument when 
> I query???
>
>
> This is my configuration:
>
>      indexed="true"  stored="true"/>
>    positionIncrementGap="100" autoGeneratePhraseQueries="true">
>      
>        
>        
>        
>                        ignoreCase="true"
>                words="lang/stopwords_en.txt"
>        />
>        generateWordParts="1" generateNumberParts="1" catenateWords="1" 
>catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
>        
>        protected="protwords.txt"/>
>        
>        
>      
>      
>        
>        ignoreCase="true" expand="true"/>
>                        ignoreCase="true"
>                words="lang/stopwords_en.txt"
>        />
>        generateWordParts="1" generateNumberParts="1" catenateWords="0" 
>catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
>        
>        protected="protwords.txt"/>
>        
>      
>    
>  stored="true" multiValued="true" omitNorms="true"/>
>
>  

Re: autoGeneratePhraseQueries not working

[Alexandre Rafalovitch]

The issue is that the Standard Query Parser does pre-processing of the
query and splits it on whitespace beforehand (to deal with all the
special syntax). So, if you don't use quoted phrases then by the time
the field specific query analyzer chain kicks in, the text is already
pre-split and the analyzer only sees one (pre space-separated) token
at a time. So, the autoGeneratePhraseQueries does not work then. If
you use different parsers that send whole text in (e.g. FieldQParser),
then - I think - it will work.

Or, like you discovered, sow=true tells the Standard Query Parser to
send it all together as well.

It is a bit of a messy part of Solr, because the Admin Analysis page
sends the text to the query analyzer without splitting (it does not
use any Query Parser). So, that adds to the confusion.

Regards,
   Alex.

On Tue, 16 Apr 2019 at 10:53, Leonardo Francalanci
 wrote:
>
>  To add some information: using "sow=true" it seems to work.But I don't 
> understand why with "sow=false" it wouldn't work (can't find anything in the 
> docs about sow interaction with autoGeneratePhraseQueries); and the 
> implication of setting saw=true.
> I've found this:[SOLR-9185] Solr's edismax and "Lucene"/standard query 
> parsers should optionally not split on whitespace before sending terms to 
> analysis - ASF JIRA
>
> |
> |
> |  |
> [SOLR-9185] Solr's edismax and "Lucene"/standard query parsers should op...
>
>
>  |
>
>  |
>
>  |
>
>
> But it's very low level and I can't find any doc more "user friendly"
>
> Il martedì 16 aprile 2019, 09:00:08 CEST, Leonardo Francalanci 
>  ha scritto:
>
>  Hi,
>
> I'm using Solr 8.0.0  I can't get autoGeneratePhraseQueries to work (also 
> tried with 7.7.1 and same result):
>
> debug":{
> "rawquerystring":"TROUBLESHOOT:my25word",
> "querystring":"TROUBLESHOOT:my25word",
> "parsedquery":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
> "parsedquery_toString":"TROUBLESHOOT:my TROUBLESHOOT:25 
> TROUBLESHOOT:word",
>
> I expected something like
>
> "parsedquery":"TROUBLESHOOT:"my 25 word"
> Why isn't autoGeneratePhraseQueries generating a quoted string argument when 
> I query???
>
>
> This is my configuration:
>
>indexed="true"  stored="true"/>
>  positionIncrementGap="100" autoGeneratePhraseQueries="true">
>   
> 
> 
> 
>  ignoreCase="true"
> words="lang/stopwords_en.txt"
> />
>  generateWordParts="1" generateNumberParts="1" catenateWords="1" 
> catenateNumbers="1" catenateAll="0" splitOnCaseChange="1"/>
> 
>  protected="protwords.txt"/>
> 
> 
>   
>   
> 
>  synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
>  ignoreCase="true"
> words="lang/stopwords_en.txt"
> />
>  generateWordParts="1" generateNumberParts="1" catenateWords="0" 
> catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/>
> 
>  protected="protwords.txt"/>
> 
>   
> 
>  stored="true" multiValued="true" omitNorms="true"/>
>
>

Re: autoGeneratePhraseQueries not working

[Leonardo Francalanci]

 To add some information: using "sow=true" it seems to work.But I don't 
understand why with "sow=false" it wouldn't work (can't find anything in the 
docs about sow interaction with autoGeneratePhraseQueries); and the implication 
of setting saw=true.
I've found this:[SOLR-9185] Solr's edismax and "Lucene"/standard query parsers 
should optionally not split on whitespace before sending terms to analysis - 
ASF JIRA

| 
| 
|  | 
[SOLR-9185] Solr's edismax and "Lucene"/standard query parsers should op...


 |

 |

 |


But it's very low level and I can't find any doc more "user friendly"

Il martedì 16 aprile 2019, 09:00:08 CEST, Leonardo Francalanci 
 ha scritto:  
 
 Hi,

I'm using Solr 8.0.0  I can't get autoGeneratePhraseQueries to work (also tried 
with 7.7.1 and same result):

debug":{
    "rawquerystring":"TROUBLESHOOT:my25word",
    "querystring":"TROUBLESHOOT:my25word",
    "parsedquery":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",
    "parsedquery_toString":"TROUBLESHOOT:my TROUBLESHOOT:25 TROUBLESHOOT:word",

I expected something like

"parsedquery":"TROUBLESHOOT:"my 25 word"
Why isn't autoGeneratePhraseQueries generating a quoted string argument when I 
query???


This is my configuration:

  
    
  
    
    
    
    
    
    
    
    
    
  
  
    
    
    
    
    
    
    
  
    


  

Re: ExactStatsCache not working for distributed IDF

[Arnold Bronley]

Hi,

I tried that as well. No change in scores.

On Thu, Mar 14, 2019 at 3:37 PM Michael Gibney 
wrote:

> Are you basing your conclusion (that it's not working as expected) on the
> scores as reported in the debug output? If you haven't already, try adding
> "score" to the "fl" param -- if different (for a given doc) than the score
> as reported in debug, then it's probably working as intended ... just a
> little confusing in the debug output.
>
> On Thu, Mar 14, 2019 at 3:23 PM Arnold Bronley 
> wrote:
>
> > Hi,
> >
> > I am using ExactStatsCache in SolrCloud (7.7.1) by adding following to
> > solrconfig.xml file for all collections. I restarted and indexed the
> > documents of all collections after this change just to be sure.
> >
> > 
> >
> > However, when I do multi-collection query, the scores do not change
> before
> > and after adding ExactStatsCache. I can still see the docCount in debug
> > output coming from individual shards and not even from whole collection.
> I
> > was expecting that the docCount would be of addition of all docCounts of
> > all collections included in search query.
> >
> > Do you know what I might be doing wrong?
> >
>

Re: ExactStatsCache not working for distributed IDF

[Michael Gibney]

Are you basing your conclusion (that it's not working as expected) on the
scores as reported in the debug output? If you haven't already, try adding
"score" to the "fl" param -- if different (for a given doc) than the score
as reported in debug, then it's probably working as intended ... just a
little confusing in the debug output.

On Thu, Mar 14, 2019 at 3:23 PM Arnold Bronley 
wrote:

> Hi,
>
> I am using ExactStatsCache in SolrCloud (7.7.1) by adding following to
> solrconfig.xml file for all collections. I restarted and indexed the
> documents of all collections after this change just to be sure.
>
> 
>
> However, when I do multi-collection query, the scores do not change before
> and after adding ExactStatsCache. I can still see the docCount in debug
> output coming from individual shards and not even from whole collection. I
> was expecting that the docCount would be of addition of all docCounts of
> all collections included in search query.
>
> Do you know what I might be doing wrong?
>

RE: SPLITSHARD not working as expected

[Oakley, Craig (NIH/NLM/NCBI) [C]]

"Sometimes for one of the sub-shards, the new leader and one of the new 
followers end up on the same instance"

Actually, it seems to be the case that every single time in the entire history 
of SPLITSHARD for one of the sub-shards, both the new leader and one of the new 
followers end up on the exact same instance.

I asked several months ago (see below under "ATTACHED MESSAGE") whether anyone 
anywhere had ever seen a case where this bug did not occur, and it seems that 
no one has been able to provide a counterexample: I think we have to concluded 
that this bug is universal

-Original Message-
From: Chris Ulicny  
Sent: Wednesday, January 30, 2019 1:46 PM
To: solr-user@lucene.apache.org
Subject: Re: SPLITSHARD not working as expected

I'm not sure what the expected behavior is. However, as of 7.4.0, it
doesn't seem like there is any attempt to prevent both the new leader and
follower replicas from being created on the same instance.

Sometimes for one of the sub-shards, the new leader and one of the new
followers end up on the same instance. We just manually end up moving them
since we don't split shards very often.

Best,
Chris

On Wed, Jan 30, 2019 at 12:46 PM Rahul Goswami 
wrote:

> Hello,
> I have a followup question on SPLITSHARD behavior. I understand that after
> a split, the leader replicas of the sub shards would reside on the same
> node as the leader of the parent. However, is there an expected behavior
> for the follower replicas of the sub shards as to where they will be
> created post split?
>
> Regards,
> Rahul
>
>
>
> On Wed, Jan 30, 2019 at 1:18 AM Rahul Goswami 
> wrote:
>
> > Thanks for the reply Jan. I have been referring to documentation for
> > SPLISHARD on 7.2.1
> > <
> https://lucene.apache.org/solr/guide/7_2/collections-api.html#splitshard>
> which
> > seems to be missing some important information present in 7.6
> > <
> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard>.
> > Especially these two pieces of information.:
> > "When using splitMethod=rewrite (default) you must ensure that the node
> > running the leader of the parent shard has enough free disk space i.e.,
> > more than twice the index size, for the split to succeed "
> >
> > "The first replicas of resulting sub-shards will always be placed on the
> > shard leader node"
> >
> > The idea of having an entire shard (both the replicas of it) present on
> > the same node did come across as an unexpected behavior at the beginning.
> > Anyway, I guess I am going to have to take care of the rebalancing with
> > MOVEREPLICA following a SPLITSHARD.
> >
> > Thanks for the clarification.
> >
> >
> > On Mon, Jan 28, 2019 at 3:40 AM Jan Høydahl 
> wrote:
> >
> >> This is normal. Please read
> >>
> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard
> >> PS: Images won't make it to the list, but don't think you need a
> >> screenshot here, what you describe is the default behaviour.
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >>
> >> > 28. jan. 2019 kl. 09:05 skrev Rahul Goswami :
> >> >
> >> > Hello,
> >> > I am using Solr 7.2.1. I created a two node example collection on the
> >> same machine. Two shards with two replicas each. I then called
> SPLITSHARD
> >> on shard2 and expected the split shards to have one replica on each
> node.
> >> However I see that for shard2_1, both replicas reside on the same node.
> Is
> >> this a valid behavior?  Unless I am missing something, this could be
> >> potentially fatal.
> >> >
> >> > Here's the query and the cluster state post split:
> >> >
> >>
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
> >> <
> >>
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
> >
> >>
> >> >
> >> >
> >> >
> >> > Thanks,
> >> > Rahul
> >>
> >>
>








 ATTACHED MESSAGE 
-Original Message-
From: Oakley, Craig (NIH/NLM/NCBI) [C]  
Sent: Wednesday, September 19, 2018 4:52 PM
To: solr-user@lucene.apache.org
Subject: RE: sharding and placement of replicas

I am still wondering whether anyone has ever seen any examples of this actually 
working (has anyone ever seen any example of SPLITSHARD on a two-node SolrCloud 
placing replicas of the each shard on different hosts than other replicas of 
the same shards

Re: SPLITSHARD not working as expected

[Chris Ulicny]

I'm not sure what the expected behavior is. However, as of 7.4.0, it
doesn't seem like there is any attempt to prevent both the new leader and
follower replicas from being created on the same instance.

Sometimes for one of the sub-shards, the new leader and one of the new
followers end up on the same instance. We just manually end up moving them
since we don't split shards very often.

Best,
Chris

On Wed, Jan 30, 2019 at 12:46 PM Rahul Goswami 
wrote:

> Hello,
> I have a followup question on SPLITSHARD behavior. I understand that after
> a split, the leader replicas of the sub shards would reside on the same
> node as the leader of the parent. However, is there an expected behavior
> for the follower replicas of the sub shards as to where they will be
> created post split?
>
> Regards,
> Rahul
>
>
>
> On Wed, Jan 30, 2019 at 1:18 AM Rahul Goswami 
> wrote:
>
> > Thanks for the reply Jan. I have been referring to documentation for
> > SPLISHARD on 7.2.1
> > <
> https://lucene.apache.org/solr/guide/7_2/collections-api.html#splitshard>
> which
> > seems to be missing some important information present in 7.6
> > <
> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard>.
> > Especially these two pieces of information.:
> > "When using splitMethod=rewrite (default) you must ensure that the node
> > running the leader of the parent shard has enough free disk space i.e.,
> > more than twice the index size, for the split to succeed "
> >
> > "The first replicas of resulting sub-shards will always be placed on the
> > shard leader node"
> >
> > The idea of having an entire shard (both the replicas of it) present on
> > the same node did come across as an unexpected behavior at the beginning.
> > Anyway, I guess I am going to have to take care of the rebalancing with
> > MOVEREPLICA following a SPLITSHARD.
> >
> > Thanks for the clarification.
> >
> >
> > On Mon, Jan 28, 2019 at 3:40 AM Jan Høydahl 
> wrote:
> >
> >> This is normal. Please read
> >>
> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard
> >> PS: Images won't make it to the list, but don't think you need a
> >> screenshot here, what you describe is the default behaviour.
> >>
> >> --
> >> Jan Høydahl, search solution architect
> >> Cominvent AS - www.cominvent.com
> >>
> >> > 28. jan. 2019 kl. 09:05 skrev Rahul Goswami :
> >> >
> >> > Hello,
> >> > I am using Solr 7.2.1. I created a two node example collection on the
> >> same machine. Two shards with two replicas each. I then called
> SPLITSHARD
> >> on shard2 and expected the split shards to have one replica on each
> node.
> >> However I see that for shard2_1, both replicas reside on the same node.
> Is
> >> this a valid behavior?  Unless I am missing something, this could be
> >> potentially fatal.
> >> >
> >> > Here's the query and the cluster state post split:
> >> >
> >>
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
> >> <
> >>
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
> >
> >>
> >> >
> >> >
> >> >
> >> > Thanks,
> >> > Rahul
> >>
> >>
>

Re: SPLITSHARD not working as expected

[Rahul Goswami]

Hello,
I have a followup question on SPLITSHARD behavior. I understand that after
a split, the leader replicas of the sub shards would reside on the same
node as the leader of the parent. However, is there an expected behavior
for the follower replicas of the sub shards as to where they will be
created post split?

Regards,
Rahul



On Wed, Jan 30, 2019 at 1:18 AM Rahul Goswami  wrote:

> Thanks for the reply Jan. I have been referring to documentation for
> SPLISHARD on 7.2.1
>  
> which
> seems to be missing some important information present in 7.6
> .
> Especially these two pieces of information.:
> "When using splitMethod=rewrite (default) you must ensure that the node
> running the leader of the parent shard has enough free disk space i.e.,
> more than twice the index size, for the split to succeed "
>
> "The first replicas of resulting sub-shards will always be placed on the
> shard leader node"
>
> The idea of having an entire shard (both the replicas of it) present on
> the same node did come across as an unexpected behavior at the beginning.
> Anyway, I guess I am going to have to take care of the rebalancing with
> MOVEREPLICA following a SPLITSHARD.
>
> Thanks for the clarification.
>
>
> On Mon, Jan 28, 2019 at 3:40 AM Jan Høydahl  wrote:
>
>> This is normal. Please read
>> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard
>> PS: Images won't make it to the list, but don't think you need a
>> screenshot here, what you describe is the default behaviour.
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>> > 28. jan. 2019 kl. 09:05 skrev Rahul Goswami :
>> >
>> > Hello,
>> > I am using Solr 7.2.1. I created a two node example collection on the
>> same machine. Two shards with two replicas each. I then called SPLITSHARD
>> on shard2 and expected the split shards to have one replica on each node.
>> However I see that for shard2_1, both replicas reside on the same node. Is
>> this a valid behavior?  Unless I am missing something, this could be
>> potentially fatal.
>> >
>> > Here's the query and the cluster state post split:
>> >
>> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
>> <
>> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true>
>>
>> >
>> >
>> >
>> > Thanks,
>> > Rahul
>>
>>

Re: SPLITSHARD not working as expected

[Rahul Goswami]

Thanks for the reply Jan. I have been referring to documentation for
SPLISHARD on 7.2.1

which
seems to be missing some important information present in 7.6
.
Especially these two pieces of information.:
"When using splitMethod=rewrite (default) you must ensure that the node
running the leader of the parent shard has enough free disk space i.e.,
more than twice the index size, for the split to succeed "

"The first replicas of resulting sub-shards will always be placed on the
shard leader node"

The idea of having an entire shard (both the replicas of it) present on the
same node did come across as an unexpected behavior at the beginning.
Anyway, I guess I am going to have to take care of the rebalancing with
MOVEREPLICA following a SPLITSHARD.

Thanks for the clarification.


On Mon, Jan 28, 2019 at 3:40 AM Jan Høydahl  wrote:

> This is normal. Please read
> https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard
> PS: Images won't make it to the list, but don't think you need a
> screenshot here, what you describe is the default behaviour.
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> > 28. jan. 2019 kl. 09:05 skrev Rahul Goswami :
> >
> > Hello,
> > I am using Solr 7.2.1. I created a two node example collection on the
> same machine. Two shards with two replicas each. I then called SPLITSHARD
> on shard2 and expected the split shards to have one replica on each node.
> However I see that for shard2_1, both replicas reside on the same node. Is
> this a valid behavior?  Unless I am missing something, this could be
> potentially fatal.
> >
> > Here's the query and the cluster state post split:
> >
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
> <
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true>
>
> >
> >
> >
> > Thanks,
> > Rahul
>
>

Re: SPLITSHARD not working as expected

[Jan Høydahl]

This is normal. Please read 
https://lucene.apache.org/solr/guide/7_6/collections-api.html#splitshard
PS: Images won't make it to the list, but don't think you need a screenshot 
here, what you describe is the default behaviour.

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 28. jan. 2019 kl. 09:05 skrev Rahul Goswami :
> 
> Hello,
> I am using Solr 7.2.1. I created a two node example collection on the same 
> machine. Two shards with two replicas each. I then called SPLITSHARD on 
> shard2 and expected the split shards to have one replica on each node. 
> However I see that for shard2_1, both replicas reside on the same node. Is 
> this a valid behavior?  Unless I am missing something, this could be 
> potentially fatal.
> 
> Here's the query and the cluster state post split:
> http://localhost:8983/solr/admin/collections?action=SPLITSHARD=gettingstarted=shard2=true
>  
> 
>  
> 
> 
>  
> Thanks,
> Rahul

Re: copyfield not working

[Jay Potharaju]

 thanks for the info Andrea!
Thanks
Jay



On Sun, Jan 13, 2019 at 11:53 PM Andrea Gazzarini 
wrote:

> Hi Jay, the text analysis always operates on the indexed content. The
> stored content of a filed is left untouched unless you do something
> before it gets indexed (e.g. on client side or by an
> UpdateRequestProcessor).
>
> Cheers,
> Andrea
>
> On 14/01/2019 08:46, Jay Potharaju wrote:
> > Hi,
> > I have a copy field in which i am copying the contents of text_en field
> to
> > another custom field.
> > After indexing i was expecting any of the special characters in the
> > paragraph to be removed, but it does not look like that is happening. The
> > copied content is same as the what is there in the source. I ran analysis
> > ...looks like the pattern matching works as expected and the special
> > characters are removed.
> >
> > Any suggestions?
> > 
>  <
> > charFilter class="solr.PatternReplaceCharFilterFactory" pattern=
> > "['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" />  > "solr.StandardTokenizerFactory"/>  > "solr.SuggestStopFilterFactory" ignoreCase="true" words=
> > "lang/stopwords_en.txt" /> 
> <
> > filter class="solr.EnglishPossessiveFilterFactory"/>  > "solr.KeywordMarkerFilterFactory" protected="protwords.txt"/>
>   > fieldType>
> >
> > Thanks
> > Jay
> >
>

Re: copyfield not working

[Jay Potharaju]

copyfield syntax from my schema file...

Thanks
Jay



On Sun, Jan 13, 2019 at 11:46 PM Jay Potharaju 
wrote:

> Hi,
> I have a copy field in which i am copying the contents of text_en field to
> another custom field.
> After indexing i was expecting any of the special characters in the
> paragraph to be removed, but it does not look like that is happening. The
> copied content is same as the what is there in the source. I ran analysis
> ...looks like the pattern matching works as expected and the special
> characters are removed.
>
> Any suggestions?
>  
>  "['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" />  "solr.StandardTokenizerFactory"/>  "solr.SuggestStopFilterFactory" ignoreCase="true" words=
> "lang/stopwords_en.txt" />  <
> filter class="solr.EnglishPossessiveFilterFactory"/>  "solr.KeywordMarkerFilterFactory" protected="protwords.txt"/> 
> 
>
> Thanks
> Jay
>
>

Re: copyfield not working

[Andrea Gazzarini]

Hi Jay, the text analysis always operates on the indexed content. The 
stored content of a filed is left untouched unless you do something 
before it gets indexed (e.g. on client side or by an 
UpdateRequestProcessor).


Cheers,
Andrea

On 14/01/2019 08:46, Jay Potharaju wrote:

Hi,
I have a copy field in which i am copying the contents of text_en field to
another custom field.
After indexing i was expecting any of the special characters in the
paragraph to be removed, but it does not look like that is happening. The
copied content is same as the what is there in the source. I ran analysis
...looks like the pattern matching works as expected and the special
characters are removed.

Any suggestions?
  <
charFilter class="solr.PatternReplaceCharFilterFactory" pattern=
"['!#\$%'\(\)\*+,-\./:;=?@\[\]\^_`{|}~!@#$%^*]" /><
filter class="solr.EnglishPossessiveFilterFactory"/>   

Thanks
Jay

Re: Dataimport not working on solrcloud

[Shawn Heisey]

On 8/20/2018 10:00 PM, Sushant Vengurlekar wrote:

I have a dataimport working on standalone solr instance but the same
doesn't work on solrcloud. I keep on hitting this error

Full Import failed:java.lang.RuntimeException:
java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException:
Exception in invoking url


There will be more to this error than what you've shared. Look in 
solr.log, and share all the ERROR/WARN entries from the correct 
timeframe.  Some of them can be quite long.  We will need *all* of that 
information.  Will also need the exact Solr version.



The url is returning well formed xml. I have verified that. The solr nodes
can fully resolve this url. I checked that out. I have the following params
set in xml-import.xml

connectionTimeout="50" readTimeout="5000"


We'll need to see the full dataimport config and the handler config from 
solrconfig.xml.


Thanks,
Shawn

Re: OR query working on Leader Node only

[Shawn Heisey]

On 8/9/2018 7:31 PM, tapan1707 wrote:

I believe correct formats for OR search query are the following:
id:(id1 or id2)
id:id1 or id:id2


The correct form for an OR query is an uppercase "OR".  The lowercase 
"or" won't do the same thing -- typically it will be interpreted as just 
another search term.


When using the edismax query parser, there is a parameter 
lowercaseOperators that (if set to true) will cause Solr to interpret 
the lowercase version as an operator.  This *only* works with the 
edismax query parser.  The default setting for lowercaseOperators was 
true before 7.0, in 7.0 the default was changed to false.


I have no idea why one node would work differently than another.  That 
sort of thing doesn't normally happen.


Thanks,
Shawn

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

Thanks for your help.

Abhi

On Thu, Mar 1, 2018 at 2:06 PM, Steve Rowe  wrote:

> Yes, either type “text_general” or some other TextField-based field type
> that includes a synonym filter.
>
> --
> Steve
> www.lucidworks.com
>
> > On Mar 1, 2018, at 3:02 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >
> > Yes, agreed. Just tested and it works. :)
> >
> > I will have a lot more fields, so every field I need a synonym feature
> for
> > will have to be type "text_general", right?
> >
> > On Thu, Mar 1, 2018 at 1:57 PM, Steve Rowe  wrote:
> >
> >> I think you want type=“text_general”
> >>
> >> --
> >> Steve
> >> www.lucidworks.com
> >>
> >>> On Mar 1, 2018, at 2:19 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >>>
> >>> Should it be defined as this instead?
> >>>
> >>>  >>> stored="true" docValues="true"/>
> >>>
> >>>
> >>>
> >>> On Thu, Mar 1, 2018 at 1:16 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >>>
>  What should PropertyAddressState type be in order to be caught into
> the
>  text_general config below?
> 
>  I have remeoved the copyfield now.
> 
>   >> stored="true" docValues="true"/>
> 
> 
> 
> 
>   >> positionIncrementGap="100">
>  
>    
> >> words="stopwords.txt" />
> >> ignoreCase="true" expand="true" tokenizerFactory="solr.
> >> StandardTokenizerFactory"/>
>    
>  
>  
>    
> >> words="stopwords.txt" />
> >> ignoreCase="true" expand="true"/>
>    
>  
>  
> 
> 
> 
>  On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:
> 
> > Hi Abhi,
> >
> > PropertyAddressState is of type “string”, which has no analysis
> >> applied.
> >
> > Since you copyfield to “text” field, which has the analysis you
> expect,
> > you could try querying it instead.
> >
> > --
> > Steve
> > www.lucidworks.com
> >
> >> On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >>
> >> Can someone please help me?
> >>
> >> Schema.xml
> >>
> >>  >> stored="true" docValues="true"/>
> >>
> >>  >> multiValued="true"/>
> >>
> >>
> >> 
> >>
> >>
> >>  >> positionIncrementGap="100">
> >> 
> >>  
> >>   >> words="stopwords.txt" />
> >>   >> ignoreCase="true" expand="true"
> >> tokenizerFactory="solr.StandardTokenizerFactory"/>
> >>  
> >> 
> >> 
> >>  
> >>   >> words="stopwords.txt" />
> >>  
> >>  
> >> 
> >> 
> >>
> >>
> >>
> >>
> >> Synonyms.txt has been populated with State abbreviations and names.
> >>
> >>
> >> When searching for
> >>
> >> PropertyAddressState:"Oregon", I do not find docs with "OR".
> >>
> >>
> >>
> >> What am I missing?
> >>
> >>
> >> Thanks,
> >>
> >> Abhi
> >
> >
> 
> 
>  --
>  Abhi Basu
> 
> >>>
> >>>
> >>>
> >>> --
> >>> Abhi Basu
> >>
> >>
> >
> >
> > --
> > Abhi Basu
>
>


-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Steve Rowe]

Yes, either type “text_general” or some other TextField-based field type that 
includes a synonym filter.

--
Steve
www.lucidworks.com

> On Mar 1, 2018, at 3:02 PM, Abhi Basu <9000r...@gmail.com> wrote:
> 
> Yes, agreed. Just tested and it works. :)
> 
> I will have a lot more fields, so every field I need a synonym feature for
> will have to be type "text_general", right?
> 
> On Thu, Mar 1, 2018 at 1:57 PM, Steve Rowe  wrote:
> 
>> I think you want type=“text_general”
>> 
>> --
>> Steve
>> www.lucidworks.com
>> 
>>> On Mar 1, 2018, at 2:19 PM, Abhi Basu <9000r...@gmail.com> wrote:
>>> 
>>> Should it be defined as this instead?
>>> 
>>> >> stored="true" docValues="true"/>
>>> 
>>> 
>>> 
>>> On Thu, Mar 1, 2018 at 1:16 PM, Abhi Basu <9000r...@gmail.com> wrote:
>>> 
 What should PropertyAddressState type be in order to be caught into the
 text_general config below?
 
 I have remeoved the copyfield now.
 
 > stored="true" docValues="true"/>
 
 
 
 
 > positionIncrementGap="100">
 
   
   > words="stopwords.txt" />
   > ignoreCase="true" expand="true" tokenizerFactory="solr.
>> StandardTokenizerFactory"/>
   
 
 
   
   > words="stopwords.txt" />
   > ignoreCase="true" expand="true"/>
   
 
 
 
 
 
 On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:
 
> Hi Abhi,
> 
> PropertyAddressState is of type “string”, which has no analysis
>> applied.
> 
> Since you copyfield to “text” field, which has the analysis you expect,
> you could try querying it instead.
> 
> --
> Steve
> www.lucidworks.com
> 
>> On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
>> 
>> Can someone please help me?
>> 
>> Schema.xml
>> 
>> > stored="true" docValues="true"/>
>> 
>> > multiValued="true"/>
>> 
>> 
>> 
>> 
>> 
>> > positionIncrementGap="100">
>> 
>>  
>>  > words="stopwords.txt" />
>>  > ignoreCase="true" expand="true"
>> tokenizerFactory="solr.StandardTokenizerFactory"/>
>>  
>> 
>> 
>>  
>>  > words="stopwords.txt" />
>>  
>>  
>> 
>> 
>> 
>> 
>> 
>> 
>> Synonyms.txt has been populated with State abbreviations and names.
>> 
>> 
>> When searching for
>> 
>> PropertyAddressState:"Oregon", I do not find docs with "OR".
>> 
>> 
>> 
>> What am I missing?
>> 
>> 
>> Thanks,
>> 
>> Abhi
> 
> 
 
 
 --
 Abhi Basu
 
>>> 
>>> 
>>> 
>>> --
>>> Abhi Basu
>> 
>> 
> 
> 
> -- 
> Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

Yes, agreed. Just tested and it works. :)

I will have a lot more fields, so every field I need a synonym feature for
will have to be type "text_general", right?

On Thu, Mar 1, 2018 at 1:57 PM, Steve Rowe  wrote:

> I think you want type=“text_general”
>
> --
> Steve
> www.lucidworks.com
>
> > On Mar 1, 2018, at 2:19 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >
> > Should it be defined as this instead?
> >
> >  > stored="true" docValues="true"/>
> >
> >
> >
> > On Thu, Mar 1, 2018 at 1:16 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >
> >> What should PropertyAddressState type be in order to be caught into the
> >> text_general config below?
> >>
> >> I have remeoved the copyfield now.
> >>
> >>  stored="true" docValues="true"/>
> >>
> >>
> >>
> >>
> >>  positionIncrementGap="100">
> >>  
> >>
> >> words="stopwords.txt" />
> >> ignoreCase="true" expand="true" tokenizerFactory="solr.
> StandardTokenizerFactory"/>
> >>
> >>  
> >>  
> >>
> >> words="stopwords.txt" />
> >> ignoreCase="true" expand="true"/>
> >>
> >>  
> >> 
> >>
> >>
> >>
> >> On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:
> >>
> >>> Hi Abhi,
> >>>
> >>> PropertyAddressState is of type “string”, which has no analysis
> applied.
> >>>
> >>> Since you copyfield to “text” field, which has the analysis you expect,
> >>> you could try querying it instead.
> >>>
> >>> --
> >>> Steve
> >>> www.lucidworks.com
> >>>
>  On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
> 
>  Can someone please help me?
> 
>  Schema.xml
> 
>    stored="true" docValues="true"/>
> 
>    multiValued="true"/>
> 
> 
>  
> 
> 
>    positionIncrementGap="100">
>  
>    
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"
>  tokenizerFactory="solr.StandardTokenizerFactory"/>
>    
>  
>  
>    
>  words="stopwords.txt" />
>    
>    
>  
>  
> 
> 
> 
> 
>  Synonyms.txt has been populated with State abbreviations and names.
> 
> 
>  When searching for
> 
>  PropertyAddressState:"Oregon", I do not find docs with "OR".
> 
> 
> 
>  What am I missing?
> 
> 
>  Thanks,
> 
>  Abhi
> >>>
> >>>
> >>
> >>
> >> --
> >> Abhi Basu
> >>
> >
> >
> >
> > --
> > Abhi Basu
>
>


-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Steve Rowe]

I think you want type=“text_general”

--
Steve
www.lucidworks.com

> On Mar 1, 2018, at 2:19 PM, Abhi Basu <9000r...@gmail.com> wrote:
> 
> Should it be defined as this instead?
> 
>  stored="true" docValues="true"/>
> 
> 
> 
> On Thu, Mar 1, 2018 at 1:16 PM, Abhi Basu <9000r...@gmail.com> wrote:
> 
>> What should PropertyAddressState type be in order to be caught into the
>> text_general config below?
>> 
>> I have remeoved the copyfield now.
>> 
>> > stored="true" docValues="true"/>
>> 
>> 
>> 
>> 
>> > positionIncrementGap="100">
>>  
>>
>>> words="stopwords.txt" />
>>> ignoreCase="true" expand="true" 
>> tokenizerFactory="solr.StandardTokenizerFactory"/>
>>
>>  
>>  
>>
>>> words="stopwords.txt" />
>>> ignoreCase="true" expand="true"/>
>>
>>  
>> 
>> 
>> 
>> 
>> On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:
>> 
>>> Hi Abhi,
>>> 
>>> PropertyAddressState is of type “string”, which has no analysis applied.
>>> 
>>> Since you copyfield to “text” field, which has the analysis you expect,
>>> you could try querying it instead.
>>> 
>>> --
>>> Steve
>>> www.lucidworks.com
>>> 
 On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
 
 Can someone please help me?
 
 Schema.xml
 
 >>> stored="true" docValues="true"/>
 
 >>> multiValued="true"/>
 
 
 
 
 
 >>> positionIncrementGap="100">
 
   
   >>> words="stopwords.txt" />
   >>> ignoreCase="true" expand="true"
 tokenizerFactory="solr.StandardTokenizerFactory"/>
   
 
 
   
   >>> words="stopwords.txt" />
   
   
 
 
 
 
 
 
 Synonyms.txt has been populated with State abbreviations and names.
 
 
 When searching for
 
 PropertyAddressState:"Oregon", I do not find docs with "OR".
 
 
 
 What am I missing?
 
 
 Thanks,
 
 Abhi
>>> 
>>> 
>> 
>> 
>> --
>> Abhi Basu
>> 
> 
> 
> 
> -- 
> Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

Should it be defined as this instead?





On Thu, Mar 1, 2018 at 1:16 PM, Abhi Basu <9000r...@gmail.com> wrote:

> What should PropertyAddressState type be in order to be caught into the
> text_general config below?
>
> I have remeoved the copyfield now.
>
>  docValues="true"/>
>
>
>
>
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true" 
> tokenizerFactory="solr.StandardTokenizerFactory"/>
> 
>   
>   
> 
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
>
>
>
> On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:
>
>> Hi Abhi,
>>
>> PropertyAddressState is of type “string”, which has no analysis applied.
>>
>> Since you copyfield to “text” field, which has the analysis you expect,
>> you could try querying it instead.
>>
>> --
>> Steve
>> www.lucidworks.com
>>
>> > On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
>> >
>> > Can someone please help me?
>> >
>> > Schema.xml
>> >
>> > > > stored="true" docValues="true"/>
>> >
>> > > > multiValued="true"/>
>> >
>> >
>> > 
>> >
>> >
>> > > > positionIncrementGap="100">
>> >  
>> >
>> >> > words="stopwords.txt" />
>> >> > ignoreCase="true" expand="true"
>> > tokenizerFactory="solr.StandardTokenizerFactory"/>
>> >
>> >  
>> >  
>> >
>> >> > words="stopwords.txt" />
>> >
>> >
>> >  
>> > 
>> >
>> >
>> >
>> >
>> > Synonyms.txt has been populated with State abbreviations and names.
>> >
>> >
>> > When searching for
>> >
>> > PropertyAddressState:"Oregon", I do not find docs with "OR".
>> >
>> >
>> >
>> > What am I missing?
>> >
>> >
>> > Thanks,
>> >
>> > Abhi
>>
>>
>
>
> --
> Abhi Basu
>



-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

What should PropertyAddressState type be in order to be caught into the
text_general config below?

I have remeoved the copyfield now.







  




  
  




  




On Thu, Mar 1, 2018 at 1:12 PM, Steve Rowe  wrote:

> Hi Abhi,
>
> PropertyAddressState is of type “string”, which has no analysis applied.
>
> Since you copyfield to “text” field, which has the analysis you expect,
> you could try querying it instead.
>
> --
> Steve
> www.lucidworks.com
>
> > On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
> >
> > Can someone please help me?
> >
> > Schema.xml
> >
> >  > stored="true" docValues="true"/>
> >
> >  > multiValued="true"/>
> >
> >
> > 
> >
> >
> >  > positionIncrementGap="100">
> >  
> >
> > > words="stopwords.txt" />
> > > ignoreCase="true" expand="true"
> > tokenizerFactory="solr.StandardTokenizerFactory"/>
> >
> >  
> >  
> >
> > > words="stopwords.txt" />
> >
> >
> >  
> > 
> >
> >
> >
> >
> > Synonyms.txt has been populated with State abbreviations and names.
> >
> >
> > When searching for
> >
> > PropertyAddressState:"Oregon", I do not find docs with "OR".
> >
> >
> >
> > What am I missing?
> >
> >
> > Thanks,
> >
> > Abhi
>
>


-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Steve Rowe]

Hi Abhi,

PropertyAddressState is of type “string”, which has no analysis applied.

Since you copyfield to “text” field, which has the analysis you expect, you 
could try querying it instead.

--
Steve
www.lucidworks.com

> On Mar 1, 2018, at 12:23 PM, Abhi Basu <9000r...@gmail.com> wrote:
> 
> Can someone please help me?
> 
> Schema.xml
> 
>  stored="true" docValues="true"/>
> 
>  multiValued="true"/>
> 
> 
> 
> 
> 
>  positionIncrementGap="100">
>  
>
> words="stopwords.txt" />
> ignoreCase="true" expand="true"
> tokenizerFactory="solr.StandardTokenizerFactory"/>
>
>  
>  
>
> words="stopwords.txt" />
>
>
>  
> 
> 
> 
> 
> 
> Synonyms.txt has been populated with State abbreviations and names.
> 
> 
> When searching for
> 
> PropertyAddressState:"Oregon", I do not find docs with "OR".
> 
> 
> 
> What am I missing?
> 
> 
> Thanks,
> 
> Abhi

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

I am testing the index analyzer first. Do I need to turn on the query
analyzer too?

synonyms.txt

Alabama, AL
Alaska, AK
Arizona, AZ
Arkansas, AR
California, CA
Colorado, CO
Connecticut, CT
Delaware, DE
Florida, FL
Georgia, GA
Hawaii, HI
Idaho, ID
Illinois, IL
Indiana, IN
Iowa, IA

etc ...


On Thu, Mar 1, 2018 at 12:27 PM, Alessandro Hoss  wrote:

> How's your synonyms declared in the file?
>
> That xml comment () in the synonym filter section isn't there in
> your running solr schema.xml, right? :)
>
> On Thu, Mar 1, 2018 at 2:53 PM Abhi Basu <9000r...@gmail.com> wrote:
>
> > Yes have tested with PA and NY, nothing works.
> >
> > On Thu, Mar 1, 2018 at 11:38 AM, Alessandro Hoss 
> > wrote:
> >
> > > Have you tested with another state?
> > >
> > > I'm asking because maybe solr is considering "OR" as a clause separator
> > > instead of a search term, and in this case the problem is not with
> > synonym,
> > > it is with your query.
> > >
> > > On Thu, Mar 1, 2018 at 2:24 PM Abhi Basu <9000r...@gmail.com> wrote:
> > >
> > > >  Can someone please help me?
> > > >
> > > > Schema.xml
> > > >
> > > >  > > > stored="true" docValues="true"/>
> > > >
> > > >  > > > multiValued="true"/>
> > > >
> > > >
> > > > 
> > > >
> > > >
> > > >  > > > positionIncrementGap="100">
> > > >   
> > > > 
> > > >  > > > words="stopwords.txt" />
> > > >  synonyms="synonyms.txt"
> > > > ignoreCase="true" expand="true"
> > > > tokenizerFactory="solr.StandardTokenizerFactory"/>
> > > > 
> > > >   
> > > >   
> > > > 
> > > >  > > > words="stopwords.txt" />
> > > > 
> > > > 
> > > >   
> > > > 
> > > >
> > > >
> > > >
> > > >
> > > > Synonyms.txt has been populated with State abbreviations and names.
> > > >
> > > >
> > > > When searching for
> > > >
> > > > PropertyAddressState:"Oregon", I do not find docs with "OR".
> > > >
> > > >
> > > >
> > > > What am I missing?
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Abhi
> > > >
> > >
> >
> >
> >
> > --
> > Abhi Basu
> >
>



-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Alessandro Hoss]

How's your synonyms declared in the file?

That xml comment () in the synonym filter section isn't there in
your running solr schema.xml, right? :)

On Thu, Mar 1, 2018 at 2:53 PM Abhi Basu <9000r...@gmail.com> wrote:

> Yes have tested with PA and NY, nothing works.
>
> On Thu, Mar 1, 2018 at 11:38 AM, Alessandro Hoss 
> wrote:
>
> > Have you tested with another state?
> >
> > I'm asking because maybe solr is considering "OR" as a clause separator
> > instead of a search term, and in this case the problem is not with
> synonym,
> > it is with your query.
> >
> > On Thu, Mar 1, 2018 at 2:24 PM Abhi Basu <9000r...@gmail.com> wrote:
> >
> > >  Can someone please help me?
> > >
> > > Schema.xml
> > >
> > >  > > stored="true" docValues="true"/>
> > >
> > >  > > multiValued="true"/>
> > >
> > >
> > > 
> > >
> > >
> > >  > > positionIncrementGap="100">
> > >   
> > > 
> > >  > > words="stopwords.txt" />
> > >  > > ignoreCase="true" expand="true"
> > > tokenizerFactory="solr.StandardTokenizerFactory"/>
> > > 
> > >   
> > >   
> > > 
> > >  > > words="stopwords.txt" />
> > > 
> > > 
> > >   
> > > 
> > >
> > >
> > >
> > >
> > > Synonyms.txt has been populated with State abbreviations and names.
> > >
> > >
> > > When searching for
> > >
> > > PropertyAddressState:"Oregon", I do not find docs with "OR".
> > >
> > >
> > >
> > > What am I missing?
> > >
> > >
> > > Thanks,
> > >
> > > Abhi
> > >
> >
>
>
>
> --
> Abhi Basu
>

Re: Synonym not working in 4.10 / CDH 5.14

[Abhi Basu]

Yes have tested with PA and NY, nothing works.

On Thu, Mar 1, 2018 at 11:38 AM, Alessandro Hoss  wrote:

> Have you tested with another state?
>
> I'm asking because maybe solr is considering "OR" as a clause separator
> instead of a search term, and in this case the problem is not with synonym,
> it is with your query.
>
> On Thu, Mar 1, 2018 at 2:24 PM Abhi Basu <9000r...@gmail.com> wrote:
>
> >  Can someone please help me?
> >
> > Schema.xml
> >
> >  > stored="true" docValues="true"/>
> >
> >  > multiValued="true"/>
> >
> >
> > 
> >
> >
> >  > positionIncrementGap="100">
> >   
> > 
> >  > words="stopwords.txt" />
> >  > ignoreCase="true" expand="true"
> > tokenizerFactory="solr.StandardTokenizerFactory"/>
> > 
> >   
> >   
> > 
> >  > words="stopwords.txt" />
> > 
> > 
> >   
> > 
> >
> >
> >
> >
> > Synonyms.txt has been populated with State abbreviations and names.
> >
> >
> > When searching for
> >
> > PropertyAddressState:"Oregon", I do not find docs with "OR".
> >
> >
> >
> > What am I missing?
> >
> >
> > Thanks,
> >
> > Abhi
> >
>



-- 
Abhi Basu

Re: Synonym not working in 4.10 / CDH 5.14

[Alessandro Hoss]

Have you tested with another state?

I'm asking because maybe solr is considering "OR" as a clause separator
instead of a search term, and in this case the problem is not with synonym,
it is with your query.

On Thu, Mar 1, 2018 at 2:24 PM Abhi Basu <9000r...@gmail.com> wrote:

>  Can someone please help me?
>
> Schema.xml
>
>  stored="true" docValues="true"/>
>
>  multiValued="true"/>
>
>
> 
>
>
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"
> tokenizerFactory="solr.StandardTokenizerFactory"/>
> 
>   
>   
> 
>  words="stopwords.txt" />
> 
> 
>   
> 
>
>
>
>
> Synonyms.txt has been populated with State abbreviations and names.
>
>
> When searching for
>
> PropertyAddressState:"Oregon", I do not find docs with "OR".
>
>
>
> What am I missing?
>
>
> Thanks,
>
> Abhi
>

Re: LTR and working with feature stores

[Dariusz Wojtas]

Ups,

Diego, I have just read your answer again.
Now I see that it is [features] element that triggers calculation of all
store features.
That gives hope model only executed the features it needs ;)

Best regards,
Dariusz Wojtas

On Sat, Jan 13, 2018 at 11:12 PM, Dariusz Wojtas  wrote:

> Hi,
>
> Thanks for the response, I understand that all features from the given
> store are calculated, no matter if they are used or not.
> OK, spread features across different models.
> But what if different models share some features?
> Creating copies of feature definitions in different stores, one per model,
> is erroneous ...
> Having several models in one store, some of them use only part of these
> features - that seems 'expensive' ;)
>
> Simple syntax evolution would be very helpful, to give {!ltr} optional
> 'store' parameter. It could override the current features store, is
> specified.
>   {!ltr reRankDocs=25 store=storeA model=simpleModelA}
>
> And {!ltr} executes 'model based calculation', not 'store based
> calculation'. Model knows what featues are required.
> Why are all features executed?
>
> Best regards,
> Dariusz Wojtas
>
>
> On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
> diego.ceccare...@gmail.com> wrote:
>
>> Hi Dariusz,
>>
>> On Jan 12, 2018 14:40, "Dariusz Wojtas"  wrote:
>>
>> Hi,
>>
>> I am working with the LTR rescoring.
>> Works beautifully, but I am curious about something.
>> How do I specify the feature store in a way different than using the
>> [features] syntax?
>> [features store=yourFeatureStore]
>>
>>
>>
>> What is the problem with this syntax? If the problem is the name of the
>> field, you can also call it by doing fl=title,authors,myfield=[features
>> store=yourFeatureStore]
>> I can't think of alternative ways..
>>
>>
>>
>> I have a range of models in my custom feature store, with plenty of
>> features implemented.
>> I have found that when I call LTR with model using only two features, Solr
>> still executes them all.
>>
>> My setup in solrconfig.xml
>> -
>> id,score,why_score:[explain style=nl],[features
>> store=store_incidentDB]
>> {!ltr reRankDocs=$reRankDocs model=simpleModelA}
>> --
>>
>> simpleModel above only uses LinearModel with 2 features.
>>
>>
>> What do I see in results?
>> In response I can see it has executed ALL features (there are values
>> calculated) in section:
>> 1)  -> response -> result -> doc -> HERE
>>
>> In addition, there is my model executed and only TWO features of the
>> executed model are presented in:
>>
>>
>> It is intended, the reason is that usually you want to execute your model
>> and at the same time log a *superset* of the features to train the next
>> model. If you want to compute only the features of the model you can
>> define
>> a featureStore that matches exactly the features that you have in the
>> model.
>>
>> 2)  -> response -> debug -> explain
>>
>> Why do I see all features being executed, if the specified model only
>> contains two features?
>>
>> I tried to reduce 'fl' to:
>>   id,score,why_score:[explain style=nl]
>> and id works as expected then:
>> 1. additional features are not executed (correct)
>> 2. my model works, only two features of the selected model (correct)
>>
>> And the final questions for this long email are:
>> 1. why does it execute all features when i specify 'store'?
>> 2. how do I specify the 'store', if I have more stores, but do not want to
>> execute all their features?
>>
>>
>> Just define a feature store that matches the features that you have in the
>> model. Please note that the featureStore that you specify in fl=
>> [features]
>> field **will not** affect the reranking (the model will compute only the
>> features that are specified in the model json file), you should ask for
>> the
>> [features] only if you want to log them.
>> Please do not hesitate to ask if something is not clear ;)
>>
>> Cheers,
>> Diego
>>
>>
>>
>> Best regards,
>> Dariusz Wojtas
>>
>
>

Re: LTR and working with feature stores

[Dariusz Wojtas]

Hi,

Thanks for the response, I understand that all features from the given
store are calculated, no matter if they are used or not.
OK, spread features across different models.
But what if different models share some features?
Creating copies of feature definitions in different stores, one per model,
is erroneous ...
Having several models in one store, some of them use only part of these
features - that seems 'expensive' ;)

Simple syntax evolution would be very helpful, to give {!ltr} optional
'store' parameter. It could override the current features store, is
specified.
  {!ltr reRankDocs=25 store=storeA model=simpleModelA}

And {!ltr} executes 'model based calculation', not 'store based
calculation'. Model knows what featues are required.
Why are all features executed?

Best regards,
Dariusz Wojtas


On Sat, Jan 13, 2018 at 4:03 PM, Diego Ceccarelli <
diego.ceccare...@gmail.com> wrote:

> Hi Dariusz,
>
> On Jan 12, 2018 14:40, "Dariusz Wojtas"  wrote:
>
> Hi,
>
> I am working with the LTR rescoring.
> Works beautifully, but I am curious about something.
> How do I specify the feature store in a way different than using the
> [features] syntax?
> [features store=yourFeatureStore]
>
>
>
> What is the problem with this syntax? If the problem is the name of the
> field, you can also call it by doing fl=title,authors,myfield=[features
> store=yourFeatureStore]
> I can't think of alternative ways..
>
>
>
> I have a range of models in my custom feature store, with plenty of
> features implemented.
> I have found that when I call LTR with model using only two features, Solr
> still executes them all.
>
> My setup in solrconfig.xml
> -
> id,score,why_score:[explain style=nl],[features
> store=store_incidentDB]
> {!ltr reRankDocs=$reRankDocs model=simpleModelA}
> --
>
> simpleModel above only uses LinearModel with 2 features.
>
>
> What do I see in results?
> In response I can see it has executed ALL features (there are values
> calculated) in section:
> 1)  -> response -> result -> doc -> HERE
>
> In addition, there is my model executed and only TWO features of the
> executed model are presented in:
>
>
> It is intended, the reason is that usually you want to execute your model
> and at the same time log a *superset* of the features to train the next
> model. If you want to compute only the features of the model you can define
> a featureStore that matches exactly the features that you have in the
> model.
>
> 2)  -> response -> debug -> explain
>
> Why do I see all features being executed, if the specified model only
> contains two features?
>
> I tried to reduce 'fl' to:
>   id,score,why_score:[explain style=nl]
> and id works as expected then:
> 1. additional features are not executed (correct)
> 2. my model works, only two features of the selected model (correct)
>
> And the final questions for this long email are:
> 1. why does it execute all features when i specify 'store'?
> 2. how do I specify the 'store', if I have more stores, but do not want to
> execute all their features?
>
>
> Just define a feature store that matches the features that you have in the
> model. Please note that the featureStore that you specify in fl= [features]
> field **will not** affect the reranking (the model will compute only the
> features that are specified in the model json file), you should ask for the
> [features] only if you want to log them.
> Please do not hesitate to ask if something is not clear ;)
>
> Cheers,
> Diego
>
>
>
> Best regards,
> Dariusz Wojtas
>

Re: LTR and working with feature stores

[Diego Ceccarelli]

Hi Dariusz,

On Jan 12, 2018 14:40, "Dariusz Wojtas"  wrote:

Hi,

I am working with the LTR rescoring.
Works beautifully, but I am curious about something.
How do I specify the feature store in a way different than using the
[features] syntax?
[features store=yourFeatureStore]



What is the problem with this syntax? If the problem is the name of the
field, you can also call it by doing fl=title,authors,myfield=[features
store=yourFeatureStore]
I can't think of alternative ways..



I have a range of models in my custom feature store, with plenty of
features implemented.
I have found that when I call LTR with model using only two features, Solr
still executes them all.

My setup in solrconfig.xml
-
id,score,why_score:[explain style=nl],[features
store=store_incidentDB]
{!ltr reRankDocs=$reRankDocs model=simpleModelA}
--

simpleModel above only uses LinearModel with 2 features.


What do I see in results?
In response I can see it has executed ALL features (there are values
calculated) in section:
1)  -> response -> result -> doc -> HERE

In addition, there is my model executed and only TWO features of the
executed model are presented in:


It is intended, the reason is that usually you want to execute your model
and at the same time log a *superset* of the features to train the next
model. If you want to compute only the features of the model you can define
a featureStore that matches exactly the features that you have in the model.

2)  -> response -> debug -> explain

Why do I see all features being executed, if the specified model only
contains two features?

I tried to reduce 'fl' to:
  id,score,why_score:[explain style=nl]
and id works as expected then:
1. additional features are not executed (correct)
2. my model works, only two features of the selected model (correct)

And the final questions for this long email are:
1. why does it execute all features when i specify 'store'?
2. how do I specify the 'store', if I have more stores, but do not want to
execute all their features?


Just define a feature store that matches the features that you have in the
model. Please note that the featureStore that you specify in fl= [features]
field **will not** affect the reranking (the model will compute only the
features that are specified in the model json file), you should ask for the
[features] only if you want to log them.
Please do not hesitate to ask if something is not clear ;)

Cheers,
Diego



Best regards,
Dariusz Wojtas

Re: Query not working with DatePointField

[Tomas Fernandez Lobbe]

The query field:* doesn't work with point fields (numerics or dates), only 
exact or range queries are supported, so an equivalent query would be field:[* 
TO *]


Sent from my iPhone

> On Jun 15, 2017, at 5:24 PM, Saurabh Sethi  wrote:
> 
> Hi,
> 
> We have a fieldType specified for date. Earlier it was using TrieDateField
> and we changed it to DatePointField.
> 
>  sortMissingLast="true" precisionStep="6"/>
> 
> 
> 
> Here are the fields used in the query and one of them uses the dateType:
> 
>  stored="false" required="true" multiValued="false"/>
>  stored="false" docValues="false" />
>  stored="false" multiValued="true" />
> 
> The following query was returning correct results when the field type was
> Trie but not with Point:
> 
> field1:value1 AND ((*:* NOT field2:*) AND field3:value3)
> 
> Any idea why field2:* does not return results anymore?
> 
> Thanks,
> Saurabh

RE: Highlighter not working on some documents

[Phil Scadden]

I managed to miss that. Thanks very much. I have some very large documents. I 
will look at index size and look at posting instead.

-Original Message-
From: David Smiley [mailto:david.w.smi...@gmail.com]
Sent: Monday, 12 June 2017 2:40 p.m.
To: solr-user@lucene.apache.org
Subject: Re: Highlighter not working on some documents

Probably the most common reason is the default hl.maxAnalyzedChars -- thus your 
highlightable text might not be in the first 51200 chars of text.  The first 
Solr release with the unified highlighter had an even lower default of 10k 
chars.

On Fri, Jun 9, 2017 at 9:58 PM Phil Scadden <p.scad...@gns.cri.nz> wrote:

> Tried hard to find difference between pdfs returning no highlighter
> and ones that do for same search term.  Includes pdfs that have been
> OCRed and ones that were text to begin with. Head scratching to me.
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: Saturday, 10 June 2017 6:22 a.m.
> To: solr-user <solr-user@lucene.apache.org>
> Subject: Re: Highlighter not working on some documents
>
> Need lots more information. I.e. schema definitions, query you use,
> handler configuration and the like. Note that highlighted fields must
> have stored="true" set and likely the _text_ field doesn't. At least
> in the default schemas stored is set to false for the catch-all field.
> And you don't want to store that information anyway since it's usually
> the destination of copyField directives and you'd highlight _those_ fields.
>
> Best,
> Erick
>
> On Thu, Jun 8, 2017 at 8:37 PM, Phil Scadden <p.scad...@gns.cri.nz> wrote:
> > Do a search with:
> > fl=id,title,datasource=true=unified=50=1=p
> > re
> > ssure+AND+testing=50=0=json
> >
> > and I get back a good list of documents. However, some documents are
> returning empty fields in the highlighter. Eg, in the highlight array have:
> > "W:\\Reports\\OCR\\4272.pdf":{"_text_":[]}
> >
> > Getting this well up the list of results with good highlighted
> > matchers
> above and below this entry. Why would the highlighter be failing?
> >
> > Notice: This email and any attachments are confidential and may not
> > be
> used, published or redistributed without the prior written consent of
> the Institute of Geological and Nuclear Sciences Limited (GNS
> Science). If received in error please destroy and immediately notify
> GNS Science. Do not copy or disclose the contents.
> Notice: This email and any attachments are confidential and may not be
> used, published or redistributed without the prior written consent of
> the Institute of Geological and Nuclear Sciences Limited (GNS
> Science). If received in error please destroy and immediately notify
> GNS Science. Do not copy or disclose the contents.
>
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
Notice: This email and any attachments are confidential and may not be used, 
published or redistributed without the prior written consent of the Institute 
of Geological and Nuclear Sciences Limited (GNS Science). If received in error 
please destroy and immediately notify GNS Science. Do not copy or disclose the 
contents.

Re: Highlighter not working on some documents

[David Smiley]

Probably the most common reason is the default hl.maxAnalyzedChars -- thus
your highlightable text might not be in the first 51200 chars of text.  The
first Solr release with the unified highlighter had an even lower default
of 10k chars.

On Fri, Jun 9, 2017 at 9:58 PM Phil Scadden <p.scad...@gns.cri.nz> wrote:

> Tried hard to find difference between pdfs returning no highlighter and
> ones that do for same search term.  Includes pdfs that have been OCRed and
> ones that were text to begin with. Head scratching to me.
>
> -Original Message-
> From: Erick Erickson [mailto:erickerick...@gmail.com]
> Sent: Saturday, 10 June 2017 6:22 a.m.
> To: solr-user <solr-user@lucene.apache.org>
> Subject: Re: Highlighter not working on some documents
>
> Need lots more information. I.e. schema definitions, query you use,
> handler configuration and the like. Note that highlighted fields must have
> stored="true" set and likely the _text_ field doesn't. At least in the
> default schemas stored is set to false for the catch-all field.
> And you don't want to store that information anyway since it's usually the
> destination of copyField directives and you'd highlight _those_ fields.
>
> Best,
> Erick
>
> On Thu, Jun 8, 2017 at 8:37 PM, Phil Scadden <p.scad...@gns.cri.nz> wrote:
> > Do a search with:
> > fl=id,title,datasource=true=unified=50=1=pre
> > ssure+AND+testing=50=0=json
> >
> > and I get back a good list of documents. However, some documents are
> returning empty fields in the highlighter. Eg, in the highlight array have:
> > "W:\\Reports\\OCR\\4272.pdf":{"_text_":[]}
> >
> > Getting this well up the list of results with good highlighted matchers
> above and below this entry. Why would the highlighter be failing?
> >
> > Notice: This email and any attachments are confidential and may not be
> used, published or redistributed without the prior written consent of the
> Institute of Geological and Nuclear Sciences Limited (GNS Science). If
> received in error please destroy and immediately notify GNS Science. Do not
> copy or disclose the contents.
> Notice: This email and any attachments are confidential and may not be
> used, published or redistributed without the prior written consent of the
> Institute of Geological and Nuclear Sciences Limited (GNS Science). If
> received in error please destroy and immediately notify GNS Science. Do not
> copy or disclose the contents.
>
-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

RE: Highlighter not working on some documents

[Phil Scadden]

Tried hard to find difference between pdfs returning no highlighter and ones 
that do for same search term.  Includes pdfs that have been OCRed and ones that 
were text to begin with. Head scratching to me.

-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Saturday, 10 June 2017 6:22 a.m.
To: solr-user <solr-user@lucene.apache.org>
Subject: Re: Highlighter not working on some documents

Need lots more information. I.e. schema definitions, query you use, handler 
configuration and the like. Note that highlighted fields must have 
stored="true" set and likely the _text_ field doesn't. At least in the default 
schemas stored is set to false for the catch-all field.
And you don't want to store that information anyway since it's usually the 
destination of copyField directives and you'd highlight _those_ fields.

Best,
Erick

On Thu, Jun 8, 2017 at 8:37 PM, Phil Scadden <p.scad...@gns.cri.nz> wrote:
> Do a search with:
> fl=id,title,datasource=true=unified=50=1=pre
> ssure+AND+testing=50=0=json
>
> and I get back a good list of documents. However, some documents are 
> returning empty fields in the highlighter. Eg, in the highlight array have:
> "W:\\Reports\\OCR\\4272.pdf":{"_text_":[]}
>
> Getting this well up the list of results with good highlighted matchers above 
> and below this entry. Why would the highlighter be failing?
>
> Notice: This email and any attachments are confidential and may not be used, 
> published or redistributed without the prior written consent of the Institute 
> of Geological and Nuclear Sciences Limited (GNS Science). If received in 
> error please destroy and immediately notify GNS Science. Do not copy or 
> disclose the contents.
Notice: This email and any attachments are confidential and may not be used, 
published or redistributed without the prior written consent of the Institute 
of Geological and Nuclear Sciences Limited (GNS Science). If received in error 
please destroy and immediately notify GNS Science. Do not copy or disclose the 
contents.

RE: Highlighter not working on some documents

[Phil Scadden]

Managed-schema attached (not a default) and the solrconfig.xml. _text_ is 
stored. (not sure how else highlighting could work??).  The indexer puts the 
body text of the pdf into _text_ field. What the value be in putting it into a 
different field and then using copyField??
 Ie
 SolrInputDocument up = new SolrInputDocument();
 String content = textHandler.toString();
 up.addField("_text_",content);

 solr.add(up);

The puzzling thing for me is why are some documents producing highlights and 
others not. The highlighters in the documents that work are pulling body text 
fragments, not things stored in some other field.

-Original Message-
From: Erick Erickson [mailto:erickerick...@gmail.com]
Sent: Saturday, 10 June 2017 6:22 a.m.
To: solr-user <solr-user@lucene.apache.org>
Subject: Re: Highlighter not working on some documents

Need lots more information. I.e. schema definitions, query you use, handler 
configuration and the like. Note that highlighted fields must have 
stored="true" set and likely the _text_ field doesn't. At least in the default 
schemas stored is set to false for the catch-all field.
And you don't want to store that information anyway since it's usually the 
destination of copyField directives and you'd highlight _those_ fields.

Best,
Erick

On Thu, Jun 8, 2017 at 8:37 PM, Phil Scadden <p.scad...@gns.cri.nz> wrote:
> Do a search with:
> fl=id,title,datasource=true=unified=50=1=pre
> ssure+AND+testing=50=0=json
>
> and I get back a good list of documents. However, some documents are 
> returning empty fields in the highlighter. Eg, in the highlight array have:
> "W:\\Reports\\OCR\\4272.pdf":{"_text_":[]}
>
> Getting this well up the list of results with good highlighted matchers above 
> and below this entry. Why would the highlighter be failing?
>
> Notice: This email and any attachments are confidential and may not be used, 
> published or redistributed without the prior written consent of the Institute 
> of Geological and Nuclear Sciences Limited (GNS Science). If received in 
> error please destroy and immediately notify GNS Science. Do not copy or 
> disclose the contents.
Notice: This email and any attachments are confidential and may not be used, 
published or redistributed without the prior written consent of the Institute 
of Geological and Nuclear Sciences Limited (GNS Science). If received in error 
please destroy and immediately notify GNS Science. Do not copy or disclose the 
contents.


solrconfig.xml
Description: solrconfig.xml

Re: Highlighter not working on some documents

[Erick Erickson]

Need lots more information. I.e. schema definitions, query you use,
handler configuration and the like. Note that highlighted fields must
have stored="true" set and likely the _text_ field doesn't. At least
in the default schemas stored is set to false for the catch-all field.
And you don't want to store that information anyway since it's usually
the destination of copyField directives and you'd highlight _those_
fields.

Best,
Erick

On Thu, Jun 8, 2017 at 8:37 PM, Phil Scadden  wrote:
> Do a search with:
> fl=id,title,datasource=true=unified=50=1=pressure+AND+testing=50=0=json
>
> and I get back a good list of documents. However, some documents are 
> returning empty fields in the highlighter. Eg, in the highlight array have:
> "W:\\Reports\\OCR\\4272.pdf":{"_text_":[]}
>
> Getting this well up the list of results with good highlighted matchers above 
> and below this entry. Why would the highlighter be failing?
>
> Notice: This email and any attachments are confidential and may not be used, 
> published or redistributed without the prior written consent of the Institute 
> of Geological and Nuclear Sciences Limited (GNS Science). If received in 
> error please destroy and immediately notify GNS Science. Do not copy or 
> disclose the contents.

Re: Join not working in Solr 6.5

[mganeshs]

Thanks for bringing up performance perspective. Is there any bench mark on
join performance when number of shards is more than 10 where documents are
indexed based on router.field.

Are you suggesting instead of router.field go for streaming expressions or
use join with router.field and then go for streaming expressions ? Can you
detail out pls ?

Thanks,



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Join-not-working-in-Solr-6-5-tp4336247p4336451.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Join not working in Solr 6.5

[Erick Erickson]

this will likely be "interesting" from a performance perspective. You
might try Streaming, especially StreamingExpressions and ParallelSQL
depending on what you need this for.

Best,
Erick

On Mon, May 22, 2017 at 12:05 AM, Damien Kamerman  wrote:
> I use a router.field so docs that I join from/to are always in the same
> shard.  See
> https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud#ShardsandIndexingDatainSolrCloud-DocumentRouting
>
> There is an open ticket SOLR-8297
> https://issues.apache.org/jira/browse/SOLR-8297 Allow join query over 2
> sharded collections: enhance functionality and exception handling
>
>
>
> On 22 May 2017 at 16:01, mganeshs  wrote:
>
>> Is there any possibility of supporting joins across multiple shards in near
>> future ? How to achieve the join when our data is spread-ed across multiple
>> shards. This is very much mandatory when we need to scale out.
>>
>> Any workarounds if out-of-box possibility is not there ?
>>
>> Thanks,
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://lucene.472066.n3.
>> nabble.com/Join-not-working-in-Solr-6-5-tp4336247p4336256.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>

Re: Join not working in Solr 6.5

[Damien Kamerman]

I use a router.field so docs that I join from/to are always in the same
shard.  See
https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud#ShardsandIndexingDatainSolrCloud-DocumentRouting

There is an open ticket SOLR-8297
https://issues.apache.org/jira/browse/SOLR-8297 Allow join query over 2
sharded collections: enhance functionality and exception handling



On 22 May 2017 at 16:01, mganeshs  wrote:

> Is there any possibility of supporting joins across multiple shards in near
> future ? How to achieve the join when our data is spread-ed across multiple
> shards. This is very much mandatory when we need to scale out.
>
> Any workarounds if out-of-box possibility is not there ?
>
> Thanks,
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Join-not-working-in-Solr-6-5-tp4336247p4336256.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Join not working in Solr 6.5

[mganeshs]

Is there any possibility of supporting joins across multiple shards in near
future ? How to achieve the join when our data is spread-ed across multiple
shards. This is very much mandatory when we need to scale out. 

Any workarounds if out-of-box possibility is not there ? 

Thanks,





--
View this message in context: 
http://lucene.472066.n3.nabble.com/Join-not-working-in-Solr-6-5-tp4336247p4336256.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Join not working in Solr 6.5

[mganeshs]

Perfect !

Sorry I overlooked and missed "="

Thanks,



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Join-not-working-in-Solr-6-5-tp4336247p4336251.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Join not working in Solr 6.5

[Damien Kamerman]

Your join should be:

{!join from=id to=C_pid_s}

On 22 May 2017 at 14:07, mganeshs  wrote:

> Hi,
>
> I have following records / documents with Parent entity
>
> id,type_s,P_hid_s,P_name_s,P_pid_s
> 11,PERSON,11,Parent1,11
>
> And following records / documents with child entity
>
> id,type_s,C_hid_s,C_name_s,C_pid_s
> 12,PERSON,12,Child2,11
> 13,PERSON,13,Child3,11
> 14,PERSON,14,Child4,11
>
> Now when I try to join and get all children of parent1 whose id is
> 11,
>
> http://localhost:8983/solr/basicns/select?indent=on={!join from id to
> C_pid_s} type_s:PERSON=json
>
>
> I get following exception
>  "error":{
> "trace":"java.lang.NullPointerException\r\n\tat
> org.apache.solr.search.JoinQuery.hashCode(JoinQParserPlugin.java:525)\r\
> n\tat
> org.apache.solr.search.QueryResultKey.(QueryResultKey.java:46)\r\n\
> tat
> org.apache.solr.search.SolrIndexSearcher.getDocListC(
> SolrIndexSearcher.java:1754)\r\n\tat
> org.apache.solr.search.SolrIndexSearcher.search(
> SolrIndexSearcher.java:609)\r\n\tat
> org.apache.solr.handler.component.QueryComponent.
> process(QueryComponent.java:547)\r\n\tat
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(
> SearchHandler.java:295)\r\n\tat
> org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:173)\r\n\tat
> org.apache.solr.core.SolrCore.execute(SolrCore.java:2440)\r\n\tat
> org.apache.solr.servlet.HttpSolrCall.execute(
> HttpSolrCall.java:723)\r\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)\r\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:347)\r\n\tat
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:298)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1691)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:582)\r\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)\r\n\tat
> org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)\r\n\tat
> org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:226)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1180)\r\n\tat
> org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:512)\r\n\tat
> org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:185)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1112)\r\n\tat
> org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)\r\n\tat
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:213)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:119)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)\r\n\tat
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> RewriteHandler.java:335)\r\n\tat
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:134)\r\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:534)\r\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\r\n\tat
> org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:251)\r\n\tat
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(
> AbstractConnection.java:273)\r\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\r\n\tat
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(
> SelectChannelEndPoint.java:93)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> executeProduceConsume(ExecuteProduceConsume.java:303)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
> produceConsume(ExecuteProduceConsume.java:148)\r\n\tat
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
> ExecuteProduceConsume.java:136)\r\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:671)\r\n\tat
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:589)\r\n\tat
> java.lang.Thread.run(Thread.java:745)\r\n",
> "code":500}}
>
>
> Is there a bug in 6.5? or something going wrong. I have used basic config
> comes with example and created collection with one shard only and not using
> multiple shards.
>
> Early response will be very much appreciated
>
>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/Join-not-working-in-Solr-6-5-tp4336247.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Backup not working

[vrindavda]

I realized that Segments_1 is getting created in Shard2 and Segments_2 in
Shard1.

Backup API is looking for Segments_1 in Shard1. Please correct if I have
configured something wrongly. I have created collection using collection API
and am using data_driven_schema_configs configs.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Backup-not-working-tp4331094p4331172.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: OCR not working occasionally

[Zheng Lin Edwin Yeo]

I have found this solution in Stackoverflow from Tim Allison to be working.

http://stackoverflow.com/questions/32354209/apache-
tika-extract-scanned-pdf-files

Regards,
Edwin

On 19 March 2017 at 19:47, Zheng Lin Edwin Yeo  wrote:

> This is my settings in the PDFParser.properties file
> under tika-parsers-1.13.jar
>
> enableAutoSpace true
> extractAnnotationText true
> sortByPosition false
> suppressDuplicateOverlappingText false
> extractAcroFormContent true
> extractInlineImages true
> extractUniqueInlineImagesOnly true
> checkExtractAccessPermission false
> allowExtractionForAccessibility true
> ifXFAExtractOnlyXFA false
> catchIntermediateIOExceptions true
>
> Regards,
> Edwin
>
>
> On 19 March 2017 at 09:08, Zheng Lin Edwin Yeo 
> wrote:
>
>> Hi Rick,
>>
>> Thanks for your reply.
>> I saw this error message for the file which has a failure.
>> Am I able to index such files together with the other files which store
>> text as an image together in the same indexing threads?
>>
>>
>> 2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
>> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
>> start commit{,optimize=false,openSearcher=true,waitSearcher=true,e
>> xpungeDeletes=false,softCommit=false,prepareCommit=false}
>> 2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
>> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.SolrIndexWriter Calling
>> setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2330f07c
>> 2017-03-19 01:02:26.610 ERROR (updateExecutor-2-thread-4-processing-n:
>> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
>> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
>> x:collection1_shard1_replica2] o.a.s.u.SolrCmdDistributor
>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error from server at http://192.168.99.1:8984/solr/
>> collection1_shard1_replica1: Expected mime type application/octet-stream
>> but got text/html. 
>> 
>> 
>> Error 404 
>> 
>> 
>> HTTP ERROR: 404
>> Problem accessing /solr/collection1_shard1_replica1/update. Reason:
>> Not Found
>> 
>> 
>> 
>>
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
>> od(HttpSolrClient.java:578)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>> pSolrClient.java:279)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>> pSolrClient.java:268)
>> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient
>> .request(ConcurrentUpdateSolrClient.java:430)
>> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
>> at org.apache.solr.update.SolrCmdDistributor.doRequest(SolrCmdD
>> istributor.java:293)
>> at org.apache.solr.update.SolrCmdDistributor.lambda$submit$0(
>> SolrCmdDistributor.java:282)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at com.codahale.metrics.InstrumentedExecutorService$Instrumente
>> dRunnable.run(InstrumentedExecutorService.java:176)
>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> at java.lang.Thread.run(Unknown Source)
>>
>> 2017-03-19 01:02:26.657 INFO  (qtp1543727556-19) [c:collection1 s:shard1
>> r:core_node1 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher
>> Opening [Searcher@77e108d5[collection1_shard1_replica2] main]
>> 2017-03-19 01:02:26.658 INFO  (qtp1543727556-19) [c:collection1 s:shard1
>> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
>> end_commit_flush
>> 2017-03-19 01:02:26.658 INFO  (searcherExecutor-16-thread-1-processing-n:
>> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
>> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
>> x:collection1_shard1_replica2] o.a.s.c.QuerySenderListener
>> QuerySenderListener sending requests to 
>> Searcher@77e108d5[collection1_shard1_replica2]
>> main{ExitableDirectoryReader(UninvertingDirectoryReader(Unin
>> verting(_0(6.4.2):C3)))}
>> 2017-03-19 01:02:26.658 INFO  (searcherExecutor-16-thread-1-processing-n:
>> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
>> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
>> x:collection1_shard1_replica2] o.a.s.c.QuerySenderListener
>> QuerySenderListener done.
>> 2017-03-19 01:02:26.659 INFO  (searcherExecutor-16-thread-1-processing-n:
>> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
>> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
>> x:collection1_shard1_replica2] o.a.s.c.SolrCore
>> [collection1_shard1_replica2] Registered new searcher Searcher@77e108d5
>> [collection1_shard1_replica2] 

Re: OCR not working occasionally

[Zheng Lin Edwin Yeo]

This is my settings in the PDFParser.properties file
under tika-parsers-1.13.jar

enableAutoSpace true
extractAnnotationText true
sortByPosition false
suppressDuplicateOverlappingText false
extractAcroFormContent true
extractInlineImages true
extractUniqueInlineImagesOnly true
checkExtractAccessPermission false
allowExtractionForAccessibility true
ifXFAExtractOnlyXFA false
catchIntermediateIOExceptions true

Regards,
Edwin


On 19 March 2017 at 09:08, Zheng Lin Edwin Yeo  wrote:

> Hi Rick,
>
> Thanks for your reply.
> I saw this error message for the file which has a failure.
> Am I able to index such files together with the other files which store
> text as an image together in the same indexing threads?
>
>
> 2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
> start commit{,optimize=false,openSearcher=true,waitSearcher=true,
> expungeDeletes=false,softCommit=false,prepareCommit=false}
> 2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.SolrIndexWriter Calling
> setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2330f07c
> 2017-03-19 01:02:26.610 ERROR (updateExecutor-2-thread-4-processing-n:
> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
> x:collection1_shard1_replica2] o.a.s.u.SolrCmdDistributor
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://192.168.99.1:8984/solr/
> collection1_shard1_replica1: Expected mime type application/octet-stream
> but got text/html. 
> 
> 
> Error 404 
> 
> 
> HTTP ERROR: 404
> Problem accessing /solr/collection1_shard1_replica1/update. Reason:
> Not Found
> 
> 
> 
>
> at org.apache.solr.client.solrj.impl.HttpSolrClient.
> executeMethod(HttpSolrClient.java:578)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> HttpSolrClient.java:279)
> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
> HttpSolrClient.java:268)
> at org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(
> ConcurrentUpdateSolrClient.java:430)
> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
> at org.apache.solr.update.SolrCmdDistributor.doRequest(
> SolrCmdDistributor.java:293)
> at org.apache.solr.update.SolrCmdDistributor.lambda$
> submit$0(SolrCmdDistributor.java:282)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at com.codahale.metrics.InstrumentedExecutorService$
> InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.
> lambda$execute$0(ExecutorUtil.java:229)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
> 2017-03-19 01:02:26.657 INFO  (qtp1543727556-19) [c:collection1 s:shard1
> r:core_node1 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher
> Opening [Searcher@77e108d5[collection1_shard1_replica2] main]
> 2017-03-19 01:02:26.658 INFO  (qtp1543727556-19) [c:collection1 s:shard1
> r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
> end_commit_flush
> 2017-03-19 01:02:26.658 INFO  (searcherExecutor-16-thread-1-processing-n:
> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
> x:collection1_shard1_replica2] o.a.s.c.QuerySenderListener
> QuerySenderListener sending requests to 
> Searcher@77e108d5[collection1_shard1_replica2]
> main{ExitableDirectoryReader(UninvertingDirectoryReader(
> Uninverting(_0(6.4.2):C3)))}
> 2017-03-19 01:02:26.658 INFO  (searcherExecutor-16-thread-1-processing-n:
> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
> x:collection1_shard1_replica2] o.a.s.c.QuerySenderListener
> QuerySenderListener done.
> 2017-03-19 01:02:26.659 INFO  (searcherExecutor-16-thread-1-processing-n:
> 192.168.99.1:8983_solr x:collection1_shard1_replica2 s:shard1
> c:collection1 r:core_node1) [c:collection1 s:shard1 r:core_node1
> x:collection1_shard1_replica2] o.a.s.c.SolrCore
> [collection1_shard1_replica2] Registered new searcher Searcher@77e108d5
> [collection1_shard1_replica2] main{ExitableDirectoryReader(
> UninvertingDirectoryReader(Uninverting(_0(6.4.2):C3)))}
> 2017-03-19 01:02:26.659 INFO  (qtp1543727556-19) [c:collection1 s:shard1
> r:core_node1 x:collection1_shard1_replica2] 
> o.a.s.u.p.LogUpdateProcessorFactory
> [collection1_shard1_replica2]  webapp=/solr path=/update
> params={update.distrib=FROMLEADER=files-
> 

Re: OCR not working occasionally

[Zheng Lin Edwin Yeo]

Hi Rick,

Thanks for your reply.
I saw this error message for the file which has a failure.
Am I able to index such files together with the other files which store
text as an image together in the same indexing threads?


2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2017-03-19 01:02:26.610 INFO  (qtp1543727556-19) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2] o.a.s.u.SolrIndexWriter Calling
setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2330f07c
2017-03-19 01:02:26.610 ERROR
(updateExecutor-2-thread-4-processing-n:192.168.99.1:8983_solr
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1)
[c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2]
o.a.s.u.SolrCmdDistributor
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://192.168.99.1:8984/solr/collection1_shard1_replica1:
Expected mime type application/octet-stream but got text/html. 


Error 404 


HTTP ERROR: 404
Problem accessing /solr/collection1_shard1_replica1/update. Reason:
Not Found




at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:578)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient.request(ConcurrentUpdateSolrClient.java:430)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at
org.apache.solr.update.SolrCmdDistributor.doRequest(SolrCmdDistributor.java:293)
at
org.apache.solr.update.SolrCmdDistributor.lambda$submit$0(SolrCmdDistributor.java:282)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

2017-03-19 01:02:26.657 INFO  (qtp1543727556-19) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2] o.a.s.s.SolrIndexSearcher
Opening [Searcher@77e108d5[collection1_shard1_replica2] main]
2017-03-19 01:02:26.658 INFO  (qtp1543727556-19) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2] o.a.s.u.DirectUpdateHandler2
end_commit_flush
2017-03-19 01:02:26.658 INFO
 (searcherExecutor-16-thread-1-processing-n:192.168.99.1:8983_solr
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1)
[c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2]
o.a.s.c.QuerySenderListener QuerySenderListener sending requests to
Searcher@77e108d5[collection1_shard1_replica2]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.4.2):C3)))}
2017-03-19 01:02:26.658 INFO
 (searcherExecutor-16-thread-1-processing-n:192.168.99.1:8983_solr
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1)
[c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2]
o.a.s.c.QuerySenderListener QuerySenderListener done.
2017-03-19 01:02:26.659 INFO
 (searcherExecutor-16-thread-1-processing-n:192.168.99.1:8983_solr
x:collection1_shard1_replica2 s:shard1 c:collection1 r:core_node1)
[c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica2]
o.a.s.c.SolrCore [collection1_shard1_replica2] Registered new searcher
Searcher@77e108d5[collection1_shard1_replica2]
main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.4.2):C3)))}
2017-03-19 01:02:26.659 INFO  (qtp1543727556-19) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2]
o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica2]
 webapp=/solr path=/update
params={update.distrib=FROMLEADER=files-update-processor=true=true=true=false=
http://192.168.99.1:8983/solr/collection1_shard1_replica2/_end_point=true=javabin=2=false}{commit=}
0 49
2017-03-19 01:02:26.662 WARN  (qtp1543727556-139) [c:collection1 s:shard1
r:core_node1 x:collection1_shard1_replica2]
o.a.s.u.p.DistributedUpdateProcessor Error sending update to
http://192.168.99.1:8984/solr
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://192.168.99.1:8984/solr/collection1_shard1_replica1:
Expected mime type application/octet-stream but got text/html. 


Error 404 


HTTP ERROR: 404
Problem accessing /solr/collection1_shard1_replica1/update. Reason:
Not Found




at

Re: OCR not working occasionally

[Rick Leir]

Hi Edwin
The pdf file format can store text as an image, and then you need OCR to get 
the text. However, text is more commonly not stored as an image in the pdf, and 
then you should not use OCR to get the text.

Do you get an error message when you have a failure?
Cheers -- Rick

On March 18, 2017 12:01:17 PM EDT, Zheng Lin Edwin Yeo  
wrote:
>Hi,
>
>I'm facing the issue of that the Tesseract OCR is not able to extract
>the
>words in a PDF file in an attachment in EMLfile and index it into Solr
>occasionally? However, most of the time it can be extracted.
>
>What could be the reason that causes the file in the email attachment
>to be
>failed to extracted using OCR?
>
>I'm using Solr 6.4.2.
>
>Regards,
>Edwin

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: autoAddReplicas:true not working

[Mark Miller]

Look at the Overseer host and see if there are any relevant logs for
autoAddReplicas.

- Mark

On Mon, Oct 24, 2016 at 3:01 PM Chetas Joshi  wrote:

> Hello,
>
> I have the following configuration for the Solr cloud and a Solr collection
> This is Solr on HDFS and Solr version I am using is 5.5.0
>
> No. of hosts: 52 (Solr Cloud)
>
> shard count:   50
> replicationFactor:   1
> MaxShardsPerNode: 1
> autoAddReplicas:   true
>
> Now, one of my shards is down. Although there are two hosts which are
> available in my cloud on which a new replica could be created, it just does
> not create a replica. All 52 hosts are healthy. What could be the reason
> for this?
>
> Thanks,
>
> Chetas.
>
-- 
- Mark
about.me/markrmiller

Re: error while working solr 6.2 cdcr

[Neeraj Bhatt]

Hi Eric

With one shard and one replica  I am getting " 08976 ERROR
(qtp110456297-13) [c:multi_dc_poc s:shard1 r:core_node1
x:multi_dc_poc_shard1_replica1] o.a.s.h.RequestHandlerBase
org.apache.solr.common.SolrException: Unknown action: null
at
org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:173)"

in target logs


Any suggestion ?

Thanks

On Thu, Nov 10, 2016 at 8:17 AM, Neeraj Bhatt 
wrote:

> Hi Eric
>
> Source have only target zookeeper ensemble (only 3 ips)
>   192.169.35.53:2181,192.169.35.249:2181,192.
> 169.35.251:2181> , because of some mail formatting ip were repeated(see
> sent item attached image). Error in log files is
> Action LASTPROCESSEDVERSION sent to non-leader replica at
> org.apache.solr.handler.CdcrRequestHandler.handleLastProcess
> edVersionAction(CdcrRequestHandler.java:498)
> I will start with empty collection with  3 nodes 3 shards one replication
> factor, It seems from log that request from souce is going to non leader
> replica.
>
> Please suggest
>
> Thanks
>
>
>
> On Wed, Nov 9, 2016 at 9:23 PM, Erick Erickson 
> wrote:
>
>> Why do you have 6 zookeepers listed in the zkHost parameter? My guess
>> is that you've listed the ZK addresses for both the source and target
>> clusters. This should only be the ZK nodes for the target cluster and
>> only present on the source.
>>
>> The source already knows its own ZK addresses, it needs to know about
>> the target cluster in order to know what nodes it should send updates
>> to.
>>
>> Best,
>> Erick
>>
>> On Wed, Nov 9, 2016 at 4:44 AM, Neeraj Bhatt 
>> wrote:
>> > Hello All
>> >
>> > I have a source cluster (3 nodes, 3 shards and 3 replica) and same for
>> > target cluster. On posting of documents to source I am getting below
>> error
>> > in target solr log files when i turned log to debug level
>> >
>> >
>> >
>> >
>> > *6538 ERROR (qtp110456297-20) [c:multi_dc_poc s:shard1 r:core_node3
>> > x:multi_dc_poc_shard1_replica2] o.a.s.h.RequestHandlerBase
>> > org.apache.solr.common.SolrException: Action LASTPROCESSEDVERSION sent
>> to
>> > non-leader replicaat
>> > org.apache.solr.handler.CdcrRequestHandler.handleLastProcess
>> edVersionAction(CdcrRequestHandler.java:498)*
>> >
>> > *should I reconfigure my replicas to one in target ?*
>> >
>> > solr config of source and target is same as given in wiki
>> > https://cwiki.apache.org/confluence/pages/viewpage.action?pa
>> geId=62687462
>> >
>> > *source *
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *> > class="solr.CdcrRequestHandler"> > > name="replica"> > > name="zkHost">192.169.35.53:2181
>> > ,192.169.35.249:2181
>> > ,192.169.35.251:2181
>> > 
>> >  > > name="source">multi_dc_poc > > name="target">multi_dc_poc
>> > > name="replicator">> > name="threadPoolSize">9> > name="schedule">1000> > name="batchSize">128> > name="updateLogSynchronizer">> > name="schedule">1000  
>> *
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *source and target update log  > > class="solr.CdcrUpdateLog">  > > name="dir">${solr.ulog.dir:}  > > name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}
>> > *
>> >
>> >
>> > *target solr config*
>> >
>> > 
>> >   
>> > disabled
>> >   
>> > 
>> >
>> > 
>> >   
>> > cdcr-processor-chain
>> >   
>> > 
>> >
>> > 
>> >   
>> >   
>> > 
>> >
>> > *target status*
>> >
>> > 0> > name="QTime">0> > name="process">stopped> > name="buffer">disabled
>> >
>> > *source status*
>> >
>> > 0> > name="QTime">0> > name="process">started> > name="buffer">enabled
>>
>
>

Re: error while working solr 6.2 cdcr

[Neeraj Bhatt]

Hi Eric

Source have only target zookeeper ensemble (only 3 ips)
  192.169.35.53:2181,192.169.35.249:2181,
192.169.35.251:2181> , because of some mail formatting ip were repeated(see
sent item attached image). Error in log files is
Action LASTPROCESSEDVERSION sent to non-leader replica at
org.apache.solr.handler.CdcrRequestHandler.handleLastProcess
edVersionAction(CdcrRequestHandler.java:498)
I will start with empty collection with  3 nodes 3 shards one replication
factor, It seems from log that request from souce is going to non leader
replica.

Please suggest

Thanks



On Wed, Nov 9, 2016 at 9:23 PM, Erick Erickson 
wrote:

> Why do you have 6 zookeepers listed in the zkHost parameter? My guess
> is that you've listed the ZK addresses for both the source and target
> clusters. This should only be the ZK nodes for the target cluster and
> only present on the source.
>
> The source already knows its own ZK addresses, it needs to know about
> the target cluster in order to know what nodes it should send updates
> to.
>
> Best,
> Erick
>
> On Wed, Nov 9, 2016 at 4:44 AM, Neeraj Bhatt 
> wrote:
> > Hello All
> >
> > I have a source cluster (3 nodes, 3 shards and 3 replica) and same for
> > target cluster. On posting of documents to source I am getting below
> error
> > in target solr log files when i turned log to debug level
> >
> >
> >
> >
> > *6538 ERROR (qtp110456297-20) [c:multi_dc_poc s:shard1 r:core_node3
> > x:multi_dc_poc_shard1_replica2] o.a.s.h.RequestHandlerBase
> > org.apache.solr.common.SolrException: Action LASTPROCESSEDVERSION sent
> to
> > non-leader replicaat
> > org.apache.solr.handler.CdcrRequestHandler.handleLastProcess
> edVersionAction(CdcrRequestHandler.java:498)*
> >
> > *should I reconfigure my replicas to one in target ?*
> >
> > solr config of source and target is same as given in wiki
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pa
> geId=62687462
> >
> > *source *
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > * > class="solr.CdcrRequestHandler">  > name="replica">  > name="zkHost">192.169.35.53:2181
> > ,192.169.35.249:2181
> > ,192.169.35.251:2181
> > 
> >   > name="source">multi_dc_poc  > name="target">multi_dc_poc
>  > name="replicator"> > name="threadPoolSize">9 > name="schedule">1000 > name="batchSize">128 > name="updateLogSynchronizer"> > name="schedule">1000  
> *
> >
> >
> >
> >
> >
> >
> >
> > *source and target update log   > class="solr.CdcrUpdateLog">   > name="dir">${solr.ulog.dir:}   > name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}
> > *
> >
> >
> > *target solr config*
> >
> > 
> >   
> > disabled
> >   
> > 
> >
> > 
> >   
> > cdcr-processor-chain
> >   
> > 
> >
> > 
> >   
> >   
> > 
> >
> > *target status*
> >
> > 0 > name="QTime">0 > name="process">stopped > name="buffer">disabled
> >
> > *source status*
> >
> > 0 > name="QTime">0 > name="process">started > name="buffer">enabled
>

Re: error while working solr 6.2 cdcr

[Erick Erickson]

Why do you have 6 zookeepers listed in the zkHost parameter? My guess
is that you've listed the ZK addresses for both the source and target
clusters. This should only be the ZK nodes for the target cluster and
only present on the source.

The source already knows its own ZK addresses, it needs to know about
the target cluster in order to know what nodes it should send updates
to.

Best,
Erick

On Wed, Nov 9, 2016 at 4:44 AM, Neeraj Bhatt  wrote:
> Hello All
>
> I have a source cluster (3 nodes, 3 shards and 3 replica) and same for
> target cluster. On posting of documents to source I am getting below error
> in target solr log files when i turned log to debug level
>
>
>
>
> *6538 ERROR (qtp110456297-20) [c:multi_dc_poc s:shard1 r:core_node3
> x:multi_dc_poc_shard1_replica2] o.a.s.h.RequestHandlerBase
> org.apache.solr.common.SolrException: Action LASTPROCESSEDVERSION sent to
> non-leader replicaat
> org.apache.solr.handler.CdcrRequestHandler.handleLastProcessedVersionAction(CdcrRequestHandler.java:498)*
>
> *should I reconfigure my replicas to one in target ?*
>
> solr config of source and target is same as given in wiki
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462
>
> *source *
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> * class="solr.CdcrRequestHandler">  name="replica">  name="zkHost">192.169.35.53:2181
> ,192.169.35.249:2181
> ,192.169.35.251:2181
> 
>   name="source">multi_dc_poc  name="target">multi_dc_poc name="replicator"> name="threadPoolSize">9 name="schedule">1000 name="batchSize">128 name="updateLogSynchronizer"> name="schedule">1000  *
>
>
>
>
>
>
>
> *source and target update log   class="solr.CdcrUpdateLog">   name="dir">${solr.ulog.dir:}   name="numVersionBuckets">${solr.ulog.numVersionBuckets:65536}
> *
>
>
> *target solr config*
>
> 
>   
> disabled
>   
> 
>
> 
>   
> cdcr-processor-chain
>   
> 
>
> 
>   
>   
> 
>
> *target status*
>
> 0 name="QTime">0 name="process">stopped name="buffer">disabled
>
> *source status*
>
> 0 name="QTime">0 name="process">started name="buffer">enabled

Re: Tutorial not working for me

[Pritchett, James]

>
>
>
> From your perspective as a new user, did you find it
> anoying/frustrating/confusing that the README.txt in the films example
> required/instructed you to first create a handful of fields using a curl
> command to hit the Schema API before you could index any of the documents?
>
> https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=
> blob;f=solr/example/films/README.txt
>
>
> N
​o, I didn't find that to be a problem. In fact, in my view that's not a
bug, that's a feature -- at least from my very limited experience, it seems
like that kind of schema setup is probably pretty standard stuff when
building a SOLR core, and so including it in the example teaches you
something useful that you'll need to do pretty much right off the bat. I
don't think that I did it via curl, though ... I must have used the admin
interface, which was just simpler than copying and pasting that
hairy-looking, multiline command into a terminal. If you used the films
example as the basis for a tutorial and wrote it up in pretty HTML, you
could include screenshots, etc. That would make it completely painless.

James

Re: Tutorial not working for me

[Chris Hostetter]

: FWIW, my next step was to work with the movie example file, which worked
: perfectly and was a much, much better "getting started" intro. You could do
: worse than to build a new tutorial/getting started from this example.
: Dataset is way more fun, too -- a quality that should never be
: underestimated in a tutorial.

Thanks for that feedback James - I have one followup question if you don't 
mind...

>From your perspective as a new user, did you find it 
anoying/frustrating/confusing that the README.txt in the films example 
required/instructed you to first create a handful of fields using a curl 
command to hit the Schema API before you could index any of the documents?

https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;a=blob;f=solr/example/films/README.txt


-Hoss
http://www.lucidworks.com/

Re: Tutorial not working for me

[Pritchett, James]

FWIW, my next step was to work with the movie example file, which worked
perfectly and was a much, much better "getting started" intro. You could do
worse than to build a new tutorial/getting started from this example.
Dataset is way more fun, too -- a quality that should never be
underestimated in a tutorial.

James

On Fri, Sep 16, 2016 at 8:34 PM, Chris Hostetter 
wrote:

>
> : I apologize if this is a really stupid question. I followed all
>
> It's not a stupid question, the tutorial is completley broken -- and for
> that matter, in my opinion, the data_driven_schema_configs used by that
> tutorial (and recommended for new users) are largely useless for the same
> underlying reason...
>
> https://issues.apache.org/jira/browse/SOLR-9526
>
> Thank you very much for asking about this - hopefully the folks who
> understand this more (and don't share my opinion that the entire concept
> of data_driven schemas are a terrible idea) can chime in and explain WTF
> is going on here)
>
>
> -Hoss
> http://www.lucidworks.com/
>



-- 


*James Pritchett*

Leader, Process Redesign and Analysis

__


*Learning Ally™*Together It’s Possible
20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608

jpritch...@learningally.org

www.LearningAlly.org 

Join us in building a community that helps blind, visually impaired &
dyslexic students thrive.

Connect with our community: *Facebook*
 | *Twitter*
 | *LinkedIn*
 |
*Explore1in5*  | *Instagram*
 | *Sign up for our community
newsletter* 

Support us: *Donate*
 | *Volunteer*

Re: Tutorial not working for me

[Steve Rowe]

For now, I was thinking of making it reflect current reality as much as 
possible, without changing coverage.

--
Steve
www.lucidworks.com

> On Sep 19, 2016, at 1:13 PM, Alexandre Rafalovitch  wrote:
> 
> Whatever works. If JIRA, SOLR-6871 is probably a reasonable place.
> Depends on the scope of "updating" you want to do.
> 
> Regards,
>   Alex.
> 
> Newsletter and resources for Solr beginners and intermediates:
> http://www.solr-start.com/
> 
> 
> On 20 September 2016 at 00:02, Steve Rowe  wrote:
>> Hi Alex,
>> 
>> Sure - I assume you mean independently from SOLR-9526 and SOLR-6871?
>> 
>> --
>> Steve
>> www.lucidworks.com
>> 
>>> On Sep 19, 2016, at 12:40 PM, Alexandre Rafalovitch  
>>> wrote:
>>> 
>>> On 19 September 2016 at 23:37, Steve Rowe  wrote:
 I’m going to start working on updating the quick start tutorial - nobody 
 has updated it since 5.0 AFAICT.
>>> 
>>> Is that something that's worth discussing in a group/JIRA/etc?
>>> 
>>> Regards,
>>>  Alex.
>>> 
>>> 
>>> Newsletter and resources for Solr beginners and intermediates:
>>> http://www.solr-start.com/
>> 

Re: Tutorial not working for me

[Alexandre Rafalovitch]

And bringing shipped and website version in sync I assume. Ok, I'll be
happy to watch whatever issue that happens in and proof-read when it
is done.

Regards,
   Alex.

Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 20 September 2016 at 00:16, Steve Rowe  wrote:
> For now, I was thinking of making it reflect current reality as much as 
> possible, without changing coverage.
>
> --
> Steve
> www.lucidworks.com
>
>> On Sep 19, 2016, at 1:13 PM, Alexandre Rafalovitch  
>> wrote:
>>
>> Whatever works. If JIRA, SOLR-6871 is probably a reasonable place.
>> Depends on the scope of "updating" you want to do.
>>
>> Regards,
>>   Alex.
>> 
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>>
>>
>> On 20 September 2016 at 00:02, Steve Rowe  wrote:
>>> Hi Alex,
>>>
>>> Sure - I assume you mean independently from SOLR-9526 and SOLR-6871?
>>>
>>> --
>>> Steve
>>> www.lucidworks.com
>>>
 On Sep 19, 2016, at 12:40 PM, Alexandre Rafalovitch  
 wrote:

 On 19 September 2016 at 23:37, Steve Rowe  wrote:
> I’m going to start working on updating the quick start tutorial - nobody 
> has updated it since 5.0 AFAICT.

 Is that something that's worth discussing in a group/JIRA/etc?

 Regards,
  Alex.

 
 Newsletter and resources for Solr beginners and intermediates:
 http://www.solr-start.com/
>>>
>

Re: Tutorial not working for me

[Alexandre Rafalovitch]

Whatever works. If JIRA, SOLR-6871 is probably a reasonable place.
Depends on the scope of "updating" you want to do.

Regards,
   Alex.

Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 20 September 2016 at 00:02, Steve Rowe  wrote:
> Hi Alex,
>
> Sure - I assume you mean independently from SOLR-9526 and SOLR-6871?
>
> --
> Steve
> www.lucidworks.com
>
>> On Sep 19, 2016, at 12:40 PM, Alexandre Rafalovitch  
>> wrote:
>>
>> On 19 September 2016 at 23:37, Steve Rowe  wrote:
>>> I’m going to start working on updating the quick start tutorial - nobody 
>>> has updated it since 5.0 AFAICT.
>>
>> Is that something that's worth discussing in a group/JIRA/etc?
>>
>> Regards,
>>   Alex.
>>
>> 
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>

Re: Tutorial not working for me

[Steve Rowe]

Hi Alex,

Sure - I assume you mean independently from SOLR-9526 and SOLR-6871?

--
Steve
www.lucidworks.com

> On Sep 19, 2016, at 12:40 PM, Alexandre Rafalovitch  
> wrote:
> 
> On 19 September 2016 at 23:37, Steve Rowe  wrote:
>> I’m going to start working on updating the quick start tutorial - nobody has 
>> updated it since 5.0 AFAICT.
> 
> Is that something that's worth discussing in a group/JIRA/etc?
> 
> Regards,
>   Alex.
> 
> 
> Newsletter and resources for Solr beginners and intermediates:
> http://www.solr-start.com/

Re: Tutorial not working for me

[Alexandre Rafalovitch]

On 19 September 2016 at 23:37, Steve Rowe  wrote:
> I’m going to start working on updating the quick start tutorial - nobody has 
> updated it since 5.0 AFAICT.

Is that something that's worth discussing in a group/JIRA/etc?

Regards,
   Alex.


Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/

Re: Tutorial not working for me

[Steve Rowe]

In the data driven configset, autoguessing text fields as the “strings" field 
type is intended to enable faceting.  The catch-all _text_ field enables search 
on all fields, but this may not be a good alternative to fielded search. 

I’m going to start working on updating the quick start tutorial - nobody has 
updated it since 5.0 AFAICT.

--
Steve
www.lucidworks.com

> On Sep 16, 2016, at 8:34 PM, Chris Hostetter  wrote:
> 
> 
> : I apologize if this is a really stupid question. I followed all
> 
> It's not a stupid question, the tutorial is completley broken -- and for 
> that matter, in my opinion, the data_driven_schema_configs used by that 
> tutorial (and recommended for new users) are largely useless for the same 
> underlying reason...
> 
> https://issues.apache.org/jira/browse/SOLR-9526
> 
> Thank you very much for asking about this - hopefully the folks who 
> understand this more (and don't share my opinion that the entire concept 
> of data_driven schemas are a terrible idea) can chime in and explain WTF 
> is going on here)
> 
> 
> -Hoss
> http://www.lucidworks.com/

Re: Tutorial not working for me

[Chris Hostetter]

: I apologize if this is a really stupid question. I followed all

It's not a stupid question, the tutorial is completley broken -- and for 
that matter, in my opinion, the data_driven_schema_configs used by that 
tutorial (and recommended for new users) are largely useless for the same 
underlying reason...

https://issues.apache.org/jira/browse/SOLR-9526

Thank you very much for asking about this - hopefully the folks who 
understand this more (and don't share my opinion that the entire concept 
of data_driven schemas are a terrible idea) can chime in and explain WTF 
is going on here)


-Hoss
http://www.lucidworks.com/

Re: Tutorial not working for me

[Pritchett, James]

Thanks for that. I totally get how it is with complicated, open source
projects. And from experience, I realize that beginner-level documentation
is really hard, especially with these kinds of projects: by the time you
get to documentation, everybody involved is so expert in all the details
that they can't imagine approaching from a blank slate.

Thanks for the suggestions. Had to chuckle, though: one of your links (
quora.com) is the one that I started with. Step 1: "Download Solr, actually
do the tutorial ..."

Best wishes,

James

On Fri, Sep 16, 2016 at 1:41 PM, John Bickerstaff 
wrote:

> I totally empathize about the sense of wasted time.  On Solr in particular
> I pulled my hair out for months - and I had access to people who had been
> using it for over two years!!!
>
> For what it's worth - this is kind of how it goes with most open source
> projects in my experience.  It's painful - and - the more moving parts the
> open source project has, the more painful the learning curve (usually)...
>
> But - the good news is that's why this list is here - we're all trying to
> help each other, so feel free to ping the list sooner rather than later
> when you're frustrated.  My new rule is one hour of being blocked...  I
> used to wait days - but everyone on the list seems to really understand how
> frustrating it is to be stuck and people have really taken time to help me
> - so I'm less hesitant.  And, of course, I try to pay it forward by
> contributing as much as I can in the same way.
>
> On that note: I've been particularly focused on working with Solr in terms
> of being able to keep upgrading simple by just replacing and re-indexing so
> if you have questions on that space (Disaster Recovery, Zookeeper config,
> etc) I may be able to help - and if you're looking for "plan" for building
> and maintaining a simple solrCloud working model on Ubuntu VMs on
> VirtualBox, I can *really* help you.
>
> Off the top of my head - some places to start:
>
> http://yonik.com/getting-started-with-solr/
> https://www.quora.com/What-is-the-best-way-to-learn-SOLR
> http://blog.outerthoughts.com/2015/11/learning-solr-comprehensively/
> http://www.solr-start.com/
>
> I think everyone responsible for those links is also a frequent "helper" on
> this email forum.
>
> Also (and I'm aware it's a glass half-full thing which frequently irritates
> me, but I'll say it anyway).  Having run into this problem I'm willing to
> wager you'll never forget this particular quirk and if you see the problem
> in future, you'll know exactly what's wrong.  It shouldn't have been
> "wrong" with the example, but for my part at least - I've begun to think of
> stuff like this as just part of the learning curve because it happens
> nearly every time.
>
> Software is hard - complex projects like SOLR are hard.  It's why we get
> paid to do stuff like this.  I'm actually getting paid pretty well right
> now because Solr is recognized as difficult and I have (with many thanks to
> this list) become known as someone who "knows Solr"...
>
> It *could* and *should* be better, but open source is what it is as a
> result of the sum total of what everyone has contributed - and we're all
> happy to help you as best we can.
>
>
>
> On Fri, Sep 16, 2016 at 11:13 AM, Pritchett, James <
> jpritch...@learningally.org> wrote:
>
> > Second possibility: You've somehow indexed fields as
> > "string" type rather than one of the text based fieldTypes.
> > "string" types are not tokenized, thus a field with
> > "My dog has fleas" will fail to find "My". It'll even not match
> > "my dog has fleas" (note capital "M").
> >
> > That appears to be the issue. Searching for name:Foundation indeed
> returns
> > the expected result. I will now go find some better entry point to SOLR
> > than the tutorial, which has wasted enough of my time for one day. Any
> > suggestions would be welcome.
> >
> > James
> >
> > On Fri, Sep 16, 2016 at 11:40 AM, Erick Erickson <
> erickerick...@gmail.com>
> > wrote:
> >
> > > My bet:
> > > the fields (look in managed_schema or, possibly schema.xml)
> > > has stored="true" and indexed="false" set for the fields
> > > in question.
> > >
> > > Pretty much everyone takes a few passes before this really
> > > makes sense. "stored" means you see the results returned,
> > > "indexed" must be true before you can search on something.
> > >
> > > Second possibility: You've somehow indexed fields as
> > > "string" type rather than one of the text based fieldTypes.
> > > "string" types are not tokenized, thus a field with
> > > "My dog has fleas" will fail to find "My". It'll even not match
> > > "my dog has fleas" (note capital "M").
> > >
> > > The admin UI>>select core>>analysis page will show you
> > > lots of this kind of detail, although I admit it takes a bit to
> > > understand all the info (do un-check the "verbose" button
> > > for the nonce).
> > >
> > > Now, all that aside, please show us the field definition for

Re: Tutorial not working for me

[John Bickerstaff]

I totally empathize about the sense of wasted time.  On Solr in particular
I pulled my hair out for months - and I had access to people who had been
using it for over two years!!!

For what it's worth - this is kind of how it goes with most open source
projects in my experience.  It's painful - and - the more moving parts the
open source project has, the more painful the learning curve (usually)...

But - the good news is that's why this list is here - we're all trying to
help each other, so feel free to ping the list sooner rather than later
when you're frustrated.  My new rule is one hour of being blocked...  I
used to wait days - but everyone on the list seems to really understand how
frustrating it is to be stuck and people have really taken time to help me
- so I'm less hesitant.  And, of course, I try to pay it forward by
contributing as much as I can in the same way.

On that note: I've been particularly focused on working with Solr in terms
of being able to keep upgrading simple by just replacing and re-indexing so
if you have questions on that space (Disaster Recovery, Zookeeper config,
etc) I may be able to help - and if you're looking for "plan" for building
and maintaining a simple solrCloud working model on Ubuntu VMs on
VirtualBox, I can *really* help you.

Off the top of my head - some places to start:

http://yonik.com/getting-started-with-solr/
https://www.quora.com/What-is-the-best-way-to-learn-SOLR
http://blog.outerthoughts.com/2015/11/learning-solr-comprehensively/
http://www.solr-start.com/

I think everyone responsible for those links is also a frequent "helper" on
this email forum.

Also (and I'm aware it's a glass half-full thing which frequently irritates
me, but I'll say it anyway).  Having run into this problem I'm willing to
wager you'll never forget this particular quirk and if you see the problem
in future, you'll know exactly what's wrong.  It shouldn't have been
"wrong" with the example, but for my part at least - I've begun to think of
stuff like this as just part of the learning curve because it happens
nearly every time.

Software is hard - complex projects like SOLR are hard.  It's why we get
paid to do stuff like this.  I'm actually getting paid pretty well right
now because Solr is recognized as difficult and I have (with many thanks to
this list) become known as someone who "knows Solr"...

It *could* and *should* be better, but open source is what it is as a
result of the sum total of what everyone has contributed - and we're all
happy to help you as best we can.



On Fri, Sep 16, 2016 at 11:13 AM, Pritchett, James <
jpritch...@learningally.org> wrote:

> Second possibility: You've somehow indexed fields as
> "string" type rather than one of the text based fieldTypes.
> "string" types are not tokenized, thus a field with
> "My dog has fleas" will fail to find "My". It'll even not match
> "my dog has fleas" (note capital "M").
>
> That appears to be the issue. Searching for name:Foundation indeed returns
> the expected result. I will now go find some better entry point to SOLR
> than the tutorial, which has wasted enough of my time for one day. Any
> suggestions would be welcome.
>
> James
>
> On Fri, Sep 16, 2016 at 11:40 AM, Erick Erickson 
> wrote:
>
> > My bet:
> > the fields (look in managed_schema or, possibly schema.xml)
> > has stored="true" and indexed="false" set for the fields
> > in question.
> >
> > Pretty much everyone takes a few passes before this really
> > makes sense. "stored" means you see the results returned,
> > "indexed" must be true before you can search on something.
> >
> > Second possibility: You've somehow indexed fields as
> > "string" type rather than one of the text based fieldTypes.
> > "string" types are not tokenized, thus a field with
> > "My dog has fleas" will fail to find "My". It'll even not match
> > "my dog has fleas" (note capital "M").
> >
> > The admin UI>>select core>>analysis page will show you
> > lots of this kind of detail, although I admit it takes a bit to
> > understand all the info (do un-check the "verbose" button
> > for the nonce).
> >
> > Now, all that aside, please show us the field definition for
> > one of the fields in question and, as John mentions, the exact
> > query (I'd also ass =true to the results).
> >
> > Saying you followed the exact instructions somewhere isn't
> > really helpful. It's likely that there's something innocent-seeming
> > that was done differently. Giving the information asked for
> > will help us diagnose what's happening and, perhaps,
> > improve the docs if we can understand the mis-match.
> >
> > Best,
> > Erick
> >
> > On Fri, Sep 16, 2016 at 8:28 AM, Pritchett, James
> >  wrote:
> > > I am following the exact instructions in the tutorial: copy and pasting
> > all
> > > commands & queries from the tutorial:
> > > https://lucene.apache.org/solr/quickstart.html. Where it breaks down
> is
> > > this one:
> > >
> > > 

Re: Tutorial not working for me

[Pritchett, James]

Second possibility: You've somehow indexed fields as
"string" type rather than one of the text based fieldTypes.
"string" types are not tokenized, thus a field with
"My dog has fleas" will fail to find "My". It'll even not match
"my dog has fleas" (note capital "M").

That appears to be the issue. Searching for name:Foundation indeed returns
the expected result. I will now go find some better entry point to SOLR
than the tutorial, which has wasted enough of my time for one day. Any
suggestions would be welcome.

James

On Fri, Sep 16, 2016 at 11:40 AM, Erick Erickson 
wrote:

> My bet:
> the fields (look in managed_schema or, possibly schema.xml)
> has stored="true" and indexed="false" set for the fields
> in question.
>
> Pretty much everyone takes a few passes before this really
> makes sense. "stored" means you see the results returned,
> "indexed" must be true before you can search on something.
>
> Second possibility: You've somehow indexed fields as
> "string" type rather than one of the text based fieldTypes.
> "string" types are not tokenized, thus a field with
> "My dog has fleas" will fail to find "My". It'll even not match
> "my dog has fleas" (note capital "M").
>
> The admin UI>>select core>>analysis page will show you
> lots of this kind of detail, although I admit it takes a bit to
> understand all the info (do un-check the "verbose" button
> for the nonce).
>
> Now, all that aside, please show us the field definition for
> one of the fields in question and, as John mentions, the exact
> query (I'd also ass =true to the results).
>
> Saying you followed the exact instructions somewhere isn't
> really helpful. It's likely that there's something innocent-seeming
> that was done differently. Giving the information asked for
> will help us diagnose what's happening and, perhaps,
> improve the docs if we can understand the mis-match.
>
> Best,
> Erick
>
> On Fri, Sep 16, 2016 at 8:28 AM, Pritchett, James
>  wrote:
> > I am following the exact instructions in the tutorial: copy and pasting
> all
> > commands & queries from the tutorial:
> > https://lucene.apache.org/solr/quickstart.html. Where it breaks down is
> > this one:
> >
> > http://localhost:8983/solr/gettingstarted/select?wt=json;
> indent=true=name:foundation
> >
> > This returns no results. Tried in the web admin view as well, also tried
> > various field:value combinations to no avail. Clearly something didn't
> get
> > configured correctly, but I saw no error messages when running all the
> data
> > loads, etc. given in the tutorial.
> >
> > Sorry to be so clueless, but I don't really have anything to go on for
> > troubleshooting besides asking dumb questions.
> >
> > James
> >
> > On Fri, Sep 16, 2016 at 11:24 AM, John Bickerstaff <
> j...@johnbickerstaff.com
> >> wrote:
> >
> >> Please share the exact query syntax?
> >>
> >> Are you using a collection you built or one of the examples?
> >>
> >> On Fri, Sep 16, 2016 at 9:06 AM, Pritchett, James <
> >> jpritch...@learningally.org> wrote:
> >>
> >> > I apologize if this is a really stupid question. I followed all
> >> > instructions on installing Tutorial, got data loaded, everything works
> >> > great until I try to query with a field name -- e.g.,
> name:foundation. I
> >> > get zero results from this or any other query which specifies a field
> >> name.
> >> > Simple queries return results, and the field names are listed in those
> >> > results correctly. But if I query using names that I know are there
> and
> >> > values that I know are there, I get nothing.
> >> >
> >> > I figure this must be something basic that is not right about the way
> >> > things have gotten set up, but I am completely blocked at this point.
> I
> >> > tried blowing it all away and restarting from scratch with no luck.
> Where
> >> > should I be looking for problems here? I am running this on a MacBook,
> >> OS X
> >> > 10.9, latest JDK (1.8).
> >> >
> >> > James
> >> >
> >> > --
> >> >
> >> >
> >> > *James Pritchett*
> >> >
> >> > Leader, Process Redesign and Analysis
> >> >
> >> > __
> >> >
> >> >
> >> > *Learning Ally™*Together It’s Possible
> >> > 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
> >> >
> >> > jpritch...@learningally.org
> >> >
> >> > www.LearningAlly.org 
> >> >
> >> > Join us in building a community that helps blind, visually impaired &
> >> > dyslexic students thrive.
> >> >
> >> > Connect with our community: *Facebook*
> >> >  | *Twitter*
> >> >  | *LinkedIn*
> >> >  |
> >> > *Explore1in5*  | *Instagram*
> >> >  | *Sign up for our community
> >> > newsletter*  >> > touch/>
> >> >
> >> > Support us: 

Re: Tutorial not working for me

[Pritchett, James]

I looked at the managed-schema and it does appear that fields like "name"
were not indexed (if I'm reading this correctly):



Not sure if this is because some step was missed, the post command was done
incorrectly, or what. Tutorial says nothing about schemas or indexes. For
reference, here's the entire tutorial run, copied and pasted from my
terminal window with just a few ellipses to collapse the 4000+ document
loads and query results.

marplon:solr-6.2.0 jpritchett$ ./bin/solr start -e cloud -noprompt

Welcome to the SolrCloud example!

Starting up 2 Solr nodes for your example SolrCloud cluster.

Creating Solr home directory
/Users/jpritchett/solr-6.2.0/example/cloud/node1/solr
Cloning /Users/jpritchett/solr-6.2.0/example/cloud/node1 into
   /Users/jpritchett/solr-6.2.0/example/cloud/node2

Starting up Solr on port 8983 using command:
bin/solr start -cloud -p 8983 -s "example/cloud/node1/solr"

Waiting up to 30 seconds to see Solr running on port 8983 [-]
Started Solr server on port 8983 (pid=8216). Happy searching!


Starting up Solr on port 7574 using command:
bin/solr start -cloud -p 7574 -s "example/cloud/node2/solr" -z
localhost:9983

Waiting up to 30 seconds to see Solr running on port 7574 [/]
Started Solr server on port 7574 (pid=8408). Happy searching!


Connecting to ZooKeeper at localhost:9983 ...
Uploading
/Users/jpritchett/solr-6.2.0/server/solr/configsets/data_driven_schema_configs/conf
for config gettingstarted to ZooKeeper at localhost:9983

Creating new collection 'gettingstarted' using command:
http://localhost:8983/solr/admin/collections?action=CREATE=gettingstarted=2=2=2=gettingstarted

{
  "responseHeader":{
"status":0,
"QTime":16272},
  "success":{
"172.16.3.78:8983_solr":{
  "responseHeader":{
"status":0,
"QTime":7526},
  "core":"gettingstarted_shard1_replica2"},
"172.16.3.78:7574_solr":{
  "responseHeader":{
"status":0,
"QTime":7838},
  "core":"gettingstarted_shard2_replica1"}}}

Enabling auto soft-commits with maxTime 3 secs using the Config API

POSTing request to Config API:
http://localhost:8983/solr/gettingstarted/config
{"set-property":{"updateHandler.autoSoftCommit.maxTime":"3000"}}
Successfully set-property updateHandler.autoSoftCommit.maxTime to 3000


SolrCloud example running, please visit: http://localhost:8983/solr

marplon:solr-6.2.0 jpritchett$ bin/post -c gettingstarted docs/
java -classpath /Users/jpritchett/solr-6.2.0/dist/solr-core-6.2.0.jar
-Dauto=yes -Dc=gettingstarted -Ddata=files -Drecursive=yes
org.apache.solr.util.SimplePostTool docs/
SimplePostTool version 5.0.0
Posting files to [base] url
http://localhost:8983/solr/gettingstarted/update...
Entering auto mode. File endings considered are
xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
Entering recursive mode, max depth=999, delay=0s
Indexing directory docs (3 files, depth=0)
POSTing file index.html (text/html) to [base]/extract
POSTing file quickstart.html (text/html) to [base]/extract

[etc.]

POSTing file SolrVelocityResourceLoader.html (text/html) to [base]/extract
POSTing file VelocityResponseWriter.html (text/html) to [base]/extract
4329 files indexed.
COMMITting Solr index changes to
http://localhost:8983/solr/gettingstarted/update...
Time spent: 0:06:03.224
marplon:solr-6.2.0 jpritchett$ bin/post -c gettingstarted
example/exampledocs/*.xml
java -classpath /Users/jpritchett/solr-6.2.0/dist/solr-core-6.2.0.jar
-Dauto=yes -Dc=gettingstarted -Ddata=files
org.apache.solr.util.SimplePostTool example/exampledocs/gb18030-example.xml
example/exampledocs/hd.xml example/exampledocs/ipod_other.xml
example/exampledocs/ipod_video.xml example/exampledocs/manufacturers.xml
example/exampledocs/mem.xml example/exampledocs/money.xml
example/exampledocs/monitor.xml example/exampledocs/monitor2.xml
example/exampledocs/mp500.xml example/exampledocs/sd500.xml
example/exampledocs/solr.xml example/exampledocs/utf8-example.xml
example/exampledocs/vidcard.xml
SimplePostTool version 5.0.0
Posting files to [base] url
http://localhost:8983/solr/gettingstarted/update...
Entering auto mode. File endings considered are
xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file gb18030-example.xml (application/xml) to [base]
POSTing file hd.xml (application/xml) to [base]

[etc.]

14 files indexed.
COMMITting Solr index changes to
http://localhost:8983/solr/gettingstarted/update...
Time spent: 0:00:07.578
marplon:solr-6.2.0 jpritchett$ bin/post -c gettingstarted
example/exampledocs/books.json
java -classpath /Users/jpritchett/solr-6.2.0/dist/solr-core-6.2.0.jar
-Dauto=yes -Dc=gettingstarted -Ddata=files
org.apache.solr.util.SimplePostTool example/exampledocs/books.json
SimplePostTool version 5.0.0
Posting files to [base] url
http://localhost:8983/solr/gettingstarted/update...
Entering auto mode. File endings considered are

Re: Tutorial not working for me

[John Bickerstaff]

what happens if you issue this?  Do you see the field in question in the
results?

http://localhost:8983/solr/gettingstarted/select?wt=json=true=

*:*

On Fri, Sep 16, 2016 at 9:43 AM, Alexandre Rafalovitch 
wrote:

> If your fields are of type string, you have to match them exactly.
>
> But the general queries are probably going against _text_ or similar which
> copyFields context of all other fields without storing but tokenize with
> its own text rules.
>
> Check df parameter in solrconfig.xml or params.json.
>
> Regards,
>Alex
>
> On 16 Sep 2016 10:06 PM, "Pritchett, James" 
> wrote:
>
> I apologize if this is a really stupid question. I followed all
> instructions on installing Tutorial, got data loaded, everything works
> great until I try to query with a field name -- e.g., name:foundation. I
> get zero results from this or any other query which specifies a field name.
> Simple queries return results, and the field names are listed in those
> results correctly. But if I query using names that I know are there and
> values that I know are there, I get nothing.
>
> I figure this must be something basic that is not right about the way
> things have gotten set up, but I am completely blocked at this point. I
> tried blowing it all away and restarting from scratch with no luck. Where
> should I be looking for problems here? I am running this on a MacBook, OS X
> 10.9, latest JDK (1.8).
>
> James
>
> --
>
>
> *James Pritchett*
>
> Leader, Process Redesign and Analysis
>
> __
>
>
> *Learning Ally™*Together It’s Possible
> 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
>
> jpritch...@learningally.org
>
> www.LearningAlly.org 
>
> Join us in building a community that helps blind, visually impaired &
> dyslexic students thrive.
>
> Connect with our community: *Facebook*
>  | *Twitter*
>  | *LinkedIn*
>  |
> *Explore1in5*  | *Instagram*
>  | *Sign up for our community
> newsletter*  touch/>
>
> Support us: *Donate*
>  | *Volunteer*
>  volunteers/how-you-can-help/>
>

Re: Tutorial not working for me

[Alexandre Rafalovitch]

If your fields are of type string, you have to match them exactly.

But the general queries are probably going against _text_ or similar which
copyFields context of all other fields without storing but tokenize with
its own text rules.

Check df parameter in solrconfig.xml or params.json.

Regards,
   Alex

On 16 Sep 2016 10:06 PM, "Pritchett, James" 
wrote:

I apologize if this is a really stupid question. I followed all
instructions on installing Tutorial, got data loaded, everything works
great until I try to query with a field name -- e.g., name:foundation. I
get zero results from this or any other query which specifies a field name.
Simple queries return results, and the field names are listed in those
results correctly. But if I query using names that I know are there and
values that I know are there, I get nothing.

I figure this must be something basic that is not right about the way
things have gotten set up, but I am completely blocked at this point. I
tried blowing it all away and restarting from scratch with no luck. Where
should I be looking for problems here? I am running this on a MacBook, OS X
10.9, latest JDK (1.8).

James

--


*James Pritchett*

Leader, Process Redesign and Analysis

__


*Learning Ally™*Together It’s Possible
20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608

jpritch...@learningally.org

www.LearningAlly.org 

Join us in building a community that helps blind, visually impaired &
dyslexic students thrive.

Connect with our community: *Facebook*
 | *Twitter*
 | *LinkedIn*
 |
*Explore1in5*  | *Instagram*
 | *Sign up for our community
newsletter* 

Support us: *Donate*
 | *Volunteer*

Re: Tutorial not working for me

[Erick Erickson]

My bet:
the fields (look in managed_schema or, possibly schema.xml)
has stored="true" and indexed="false" set for the fields
in question.

Pretty much everyone takes a few passes before this really
makes sense. "stored" means you see the results returned,
"indexed" must be true before you can search on something.

Second possibility: You've somehow indexed fields as
"string" type rather than one of the text based fieldTypes.
"string" types are not tokenized, thus a field with
"My dog has fleas" will fail to find "My". It'll even not match
"my dog has fleas" (note capital "M").

The admin UI>>select core>>analysis page will show you
lots of this kind of detail, although I admit it takes a bit to
understand all the info (do un-check the "verbose" button
for the nonce).

Now, all that aside, please show us the field definition for
one of the fields in question and, as John mentions, the exact
query (I'd also ass =true to the results).

Saying you followed the exact instructions somewhere isn't
really helpful. It's likely that there's something innocent-seeming
that was done differently. Giving the information asked for
will help us diagnose what's happening and, perhaps,
improve the docs if we can understand the mis-match.

Best,
Erick

On Fri, Sep 16, 2016 at 8:28 AM, Pritchett, James
 wrote:
> I am following the exact instructions in the tutorial: copy and pasting all
> commands & queries from the tutorial:
> https://lucene.apache.org/solr/quickstart.html. Where it breaks down is
> this one:
>
> http://localhost:8983/solr/gettingstarted/select?wt=json=true=name:foundation
>
> This returns no results. Tried in the web admin view as well, also tried
> various field:value combinations to no avail. Clearly something didn't get
> configured correctly, but I saw no error messages when running all the data
> loads, etc. given in the tutorial.
>
> Sorry to be so clueless, but I don't really have anything to go on for
> troubleshooting besides asking dumb questions.
>
> James
>
> On Fri, Sep 16, 2016 at 11:24 AM, John Bickerstaff > wrote:
>
>> Please share the exact query syntax?
>>
>> Are you using a collection you built or one of the examples?
>>
>> On Fri, Sep 16, 2016 at 9:06 AM, Pritchett, James <
>> jpritch...@learningally.org> wrote:
>>
>> > I apologize if this is a really stupid question. I followed all
>> > instructions on installing Tutorial, got data loaded, everything works
>> > great until I try to query with a field name -- e.g., name:foundation. I
>> > get zero results from this or any other query which specifies a field
>> name.
>> > Simple queries return results, and the field names are listed in those
>> > results correctly. But if I query using names that I know are there and
>> > values that I know are there, I get nothing.
>> >
>> > I figure this must be something basic that is not right about the way
>> > things have gotten set up, but I am completely blocked at this point. I
>> > tried blowing it all away and restarting from scratch with no luck. Where
>> > should I be looking for problems here? I am running this on a MacBook,
>> OS X
>> > 10.9, latest JDK (1.8).
>> >
>> > James
>> >
>> > --
>> >
>> >
>> > *James Pritchett*
>> >
>> > Leader, Process Redesign and Analysis
>> >
>> > __
>> >
>> >
>> > *Learning Ally™*Together It’s Possible
>> > 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
>> >
>> > jpritch...@learningally.org
>> >
>> > www.LearningAlly.org 
>> >
>> > Join us in building a community that helps blind, visually impaired &
>> > dyslexic students thrive.
>> >
>> > Connect with our community: *Facebook*
>> >  | *Twitter*
>> >  | *LinkedIn*
>> >  |
>> > *Explore1in5*  | *Instagram*
>> >  | *Sign up for our community
>> > newsletter* > > touch/>
>> >
>> > Support us: *Donate*
>> >  | *Volunteer*
>> > > > volunteers/how-you-can-help/>
>> >
>>
>
>
>
> --
>
>
> *James Pritchett*
>
> Leader, Process Redesign and Analysis
>
> __
>
>
> *Learning Ally™*Together It’s Possible
> 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
>
> jpritch...@learningally.org
>
> www.LearningAlly.org 
>
> Join us in building a community that helps blind, visually impaired &
> dyslexic students thrive.
>
> Connect with our community: *Facebook*
>  | *Twitter*
>  | *LinkedIn*
>  |
> *Explore1in5* 

Re: Tutorial not working for me

[Pritchett, James]

I am following the exact instructions in the tutorial: copy and pasting all
commands & queries from the tutorial:
https://lucene.apache.org/solr/quickstart.html. Where it breaks down is
this one:

http://localhost:8983/solr/gettingstarted/select?wt=json=true=name:foundation

This returns no results. Tried in the web admin view as well, also tried
various field:value combinations to no avail. Clearly something didn't get
configured correctly, but I saw no error messages when running all the data
loads, etc. given in the tutorial.

Sorry to be so clueless, but I don't really have anything to go on for
troubleshooting besides asking dumb questions.

James

On Fri, Sep 16, 2016 at 11:24 AM, John Bickerstaff  wrote:

> Please share the exact query syntax?
>
> Are you using a collection you built or one of the examples?
>
> On Fri, Sep 16, 2016 at 9:06 AM, Pritchett, James <
> jpritch...@learningally.org> wrote:
>
> > I apologize if this is a really stupid question. I followed all
> > instructions on installing Tutorial, got data loaded, everything works
> > great until I try to query with a field name -- e.g., name:foundation. I
> > get zero results from this or any other query which specifies a field
> name.
> > Simple queries return results, and the field names are listed in those
> > results correctly. But if I query using names that I know are there and
> > values that I know are there, I get nothing.
> >
> > I figure this must be something basic that is not right about the way
> > things have gotten set up, but I am completely blocked at this point. I
> > tried blowing it all away and restarting from scratch with no luck. Where
> > should I be looking for problems here? I am running this on a MacBook,
> OS X
> > 10.9, latest JDK (1.8).
> >
> > James
> >
> > --
> >
> >
> > *James Pritchett*
> >
> > Leader, Process Redesign and Analysis
> >
> > __
> >
> >
> > *Learning Ally™*Together It’s Possible
> > 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
> >
> > jpritch...@learningally.org
> >
> > www.LearningAlly.org 
> >
> > Join us in building a community that helps blind, visually impaired &
> > dyslexic students thrive.
> >
> > Connect with our community: *Facebook*
> >  | *Twitter*
> >  | *LinkedIn*
> >  |
> > *Explore1in5*  | *Instagram*
> >  | *Sign up for our community
> > newsletter*  > touch/>
> >
> > Support us: *Donate*
> >  | *Volunteer*
> >  > volunteers/how-you-can-help/>
> >
>



-- 


*James Pritchett*

Leader, Process Redesign and Analysis

__


*Learning Ally™*Together It’s Possible
20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608

jpritch...@learningally.org

www.LearningAlly.org 

Join us in building a community that helps blind, visually impaired &
dyslexic students thrive.

Connect with our community: *Facebook*
 | *Twitter*
 | *LinkedIn*
 |
*Explore1in5*  | *Instagram*
 | *Sign up for our community
newsletter* 

Support us: *Donate*
 | *Volunteer*

Re: Tutorial not working for me

[John Bickerstaff]

Please share the exact query syntax?

Are you using a collection you built or one of the examples?

On Fri, Sep 16, 2016 at 9:06 AM, Pritchett, James <
jpritch...@learningally.org> wrote:

> I apologize if this is a really stupid question. I followed all
> instructions on installing Tutorial, got data loaded, everything works
> great until I try to query with a field name -- e.g., name:foundation. I
> get zero results from this or any other query which specifies a field name.
> Simple queries return results, and the field names are listed in those
> results correctly. But if I query using names that I know are there and
> values that I know are there, I get nothing.
>
> I figure this must be something basic that is not right about the way
> things have gotten set up, but I am completely blocked at this point. I
> tried blowing it all away and restarting from scratch with no luck. Where
> should I be looking for problems here? I am running this on a MacBook, OS X
> 10.9, latest JDK (1.8).
>
> James
>
> --
>
>
> *James Pritchett*
>
> Leader, Process Redesign and Analysis
>
> __
>
>
> *Learning Ally™*Together It’s Possible
> 20 Roszel Road | Princeton, NJ 08540 | Office: 609.243.7608
>
> jpritch...@learningally.org
>
> www.LearningAlly.org 
>
> Join us in building a community that helps blind, visually impaired &
> dyslexic students thrive.
>
> Connect with our community: *Facebook*
>  | *Twitter*
>  | *LinkedIn*
>  |
> *Explore1in5*  | *Instagram*
>  | *Sign up for our community
> newsletter*  touch/>
>
> Support us: *Donate*
>  | *Volunteer*
>  volunteers/how-you-can-help/>
>

Re: QParsePlugin not working on sharded collection

[tedsolr]

So my implementation with a DocTransformer is causing an exception (with a
sharded collection):

ERROR - 2016-08-04 09:41:44.247; [ShardTest1 shard1_0 core_node3
ShardTest1_shard1_0_replica1] org.apache.solr.common.SolrException;
null:org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
Error from server at
http://localhost:8983/solr/ShardTest1_shard1_0_replica1: parsing error
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:538)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:218)
at
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:183)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.common.SolrException: parsing error
at
org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:52)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:536)
... 12 more
Caused by: java.io.EOFException
at
org.apache.solr.common.util.FastInputStream.readByte(FastInputStream.java:208)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:188)
at
org.apache.solr.common.util.JavaBinCodec.readArray(JavaBinCodec.java:508)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:202)
at
org.apache.solr.common.util.JavaBinCodec.readSolrDocumentList(JavaBinCodec.java:390)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:237)
at
org.apache.solr.common.util.JavaBinCodec.readOrderedMap(JavaBinCodec.java:135)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:204)
at
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:126)
at
org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:50)
... 13 more

Here are the changes to TedQuery (I reduced the amount of data being
returned and map the docId to the document - like the [docid] transformer,
and put the map in the request context in the finish() method)

public void collect(int doc) throws IOException {
count++;
if (doc % 1 == 0) {
mydata.put(Integer.valueOf(doc + 
super.docBase), String.valueOf(doc +
super.docBase));
super.collect(doc);
}
}

public void finish() throws IOException {
...
rb.req.getContext().put("mystats", mydata);
...
}

Here's the transformer:

public class TedTransform extends TransformerFactory {
@Override
public DocTransformer create(String arg0, SolrParams arg1, 
SolrQueryRequest
arg2) {
return new TedTransformer(arg0, arg2);
}

private class TedTransformer extends TransformerWithContext {
private final String f;
private HashMap data;

public TedTransformer(String f, SolrQueryRequest r) {
this.f = f;
}

@Override
public String getName() {
return null;
}

@Override
public void transform(SolrDocument arg0, int arg1) throws 
IOException {
if (context.req != null) {
if (data == null) {
data = (HashMap)
context.req.getContext().get("mystats");
}
arg0.setField(f, data.get(Integer.valueOf(arg1)));
}
}
}
}

And I added the transformer to the solrconfig.xml:


   
   
   
{!TedFilter myvar=hello}
[TedT]

   

Why does this barf on multi-sharded collections?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/QParsePlugin-not-working-on-sharded-collection-tp4290249p4290390.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: QParsePlugin not working on sharded collection

[tedsolr]

Thanks Erick, you answered my question by pointing out the aggregator. I
didn't realize a merge strategy was _required_ to return stats info when
there are multiple shards. I'm having trouble with my actual plugin so I've
scaled back to the simplest possible example. I'm adding to it little by
little to see what the last straw is.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/QParsePlugin-not-working-on-sharded-collection-tp4290249p4290365.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: QParsePlugin not working on sharded collection

[Erick Erickson]

OK, I'm going to assume that somewhere you're
keeping more complicated structures around to
track all the docs coming through the collector so
you can know whether they're duplicates or not.

I think there are really two ways (at least) to go about
it
1> use a SearchComponent to add a separate section to
the response similar to highlighting or faceting.

2> go ahead and use a DocTransformer to add the data
to each individual doc. But the example you're using adds the
data to the meta-data, not an individual doc.


Best,
Erick

On Wed, Aug 3, 2016 at 2:03 PM, tedsolr  wrote:
> So I notice if I create the simplest MergeStrategy I can get my test values
> from the shard responses and then if I add info to the SolrQueryResponse it
> gets back to the caller. I still must be missing something. I wouldn't
> expect to have different code paths - one for single shard one for multi
> shard. So if the PostFilter is restricting the documents returned, what's
> the correct way to return my analytics info? Should I not be adding data to
> the SolrQueryResponse from within the delegating collector's finish()
> method? Here's what I'm trying to do (still works fine with a single shard
> collection :)
>
> - Use the DelegatingCollector to restrict docs returned (dropping docs that
> are "duplicates" based on my critieria)
> - Calculate 2 stats for each collected doc: a count of "duplicate" docs & a
> sum on a number field from these "duplicate" docs. I am doing the math in
> the collect() method.
> - Return the stats in the response stream. I'm using a TransformerFactory
> now to inject a new field into the results for each doc. Should I be using a
> SearchComponent instead?
>
>
> Erick Erickson wrote
>> Right, I don't have the code in front of me right now, but I think
>> your issue is at the "aggregation" point. You also have to put
>> some code in the aggregation bits that pull your custom parts
>> from the sub-request packets and puts in the final packet,
>> "doing the right thing" in terms of assembling them into
>> something meaningful along the way (e.g. averaging "myvar"
>> or putting it in a list identified by shard or..).
>>
>> I think if you fire the query at one of your shards with =false
>> you'll see your additions, which would demonstrate that your
>> filter is being found. I assume your custom jar is on the shards
>> or you'd get an exception (assuming you've pushed your
>> solrconfig to ZK).
>>
>> Best,
>> Erick
>
>
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/QParsePlugin-not-working-on-sharded-collection-tp4290249p4290285.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: QParsePlugin not working on sharded collection

[tedsolr]

So I notice if I create the simplest MergeStrategy I can get my test values
from the shard responses and then if I add info to the SolrQueryResponse it
gets back to the caller. I still must be missing something. I wouldn't
expect to have different code paths - one for single shard one for multi
shard. So if the PostFilter is restricting the documents returned, what's
the correct way to return my analytics info? Should I not be adding data to
the SolrQueryResponse from within the delegating collector's finish()
method? Here's what I'm trying to do (still works fine with a single shard
collection :)

- Use the DelegatingCollector to restrict docs returned (dropping docs that
are "duplicates" based on my critieria)
- Calculate 2 stats for each collected doc: a count of "duplicate" docs & a
sum on a number field from these "duplicate" docs. I am doing the math in
the collect() method.
- Return the stats in the response stream. I'm using a TransformerFactory
now to inject a new field into the results for each doc. Should I be using a
SearchComponent instead?


Erick Erickson wrote
> Right, I don't have the code in front of me right now, but I think
> your issue is at the "aggregation" point. You also have to put
> some code in the aggregation bits that pull your custom parts
> from the sub-request packets and puts in the final packet,
> "doing the right thing" in terms of assembling them into
> something meaningful along the way (e.g. averaging "myvar"
> or putting it in a list identified by shard or..).
> 
> I think if you fire the query at one of your shards with =false
> you'll see your additions, which would demonstrate that your
> filter is being found. I assume your custom jar is on the shards
> or you'd get an exception (assuming you've pushed your
> solrconfig to ZK).
> 
> Best,
> Erick





--
View this message in context: 
http://lucene.472066.n3.nabble.com/QParsePlugin-not-working-on-sharded-collection-tp4290249p4290285.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: QParsePlugin not working on sharded collection

[Erick Erickson]

Right, I don't have the code in front of me right now, but I think
your issue is at the "aggregation" point. You also have to put
some code in the aggregation bits that pull your custom parts
from the sub-request packets and puts in the final packet,
"doing the right thing" in terms of assembling them into
something meaningful along the way (e.g. averaging "myvar"
or putting it in a list identified by shard or..).

I think if you fire the query at one of your shards with =false
you'll see your additions, which would demonstrate that your
filter is being found. I assume your custom jar is on the shards
or you'd get an exception (assuming you've pushed your
solrconfig to ZK).

Best,
Erick

On Wed, Aug 3, 2016 at 9:42 AM, tedsolr  wrote:
> I'm trying to verify that a very simple custom post filter will work on a
> sharded collection. So far it doesn't. Here are the search results on my
> single shard test collection:
>
> {
>   "responseHeader": {
> "status": 0,
> "QTime": 17
>   },
>   "thecountis": "946028",
>   "myvar": "hello",
>   "response": {
> "numFound": 946028,
> "start": 0,
> "docs": [
> ...]
> }
>
> When I run against a two shard collection (same data set) it's as though the
> post filter doesn't exist. The results don't include my additions to the
> response:
>
> {
>   "responseHeader": {
> "status": 0,
> "QTime": 17
>   },
>   "response": {
> "numFound": 946028,
> "start": 0,
> "docs": [
> ...]
> }
>
> Here's the solconfig.xml:
>
> ...
> 
>
>
> {!TedFilter myvar=hello}
> 
>
> ...
>
> And here's the simplest plugin I could write:
>
> public class TedPlugin extends QParserPlugin {
> @Override
> public void init(NamedList arg0) {
> }
>
> @Override
> public QParser createParser(String arg0, final SolrParams arg1, final
> SolrParams arg2, final SolrQueryRequest arg3) {
> return new QParser(arg0, arg1, arg2, arg3) {
>
> @Override
> public Query parse() throws SyntaxError {
> return new TedQuery(arg1, arg2, arg3);
> }
> };
> }
> }
>
> public class TedQuery extends AnalyticsQuery {
> private final String myvar;
>
> TedQuery(SolrParams localParams, SolrParams params, SolrQueryRequest 
> req) {
> myvar = localParams.get("myvar");
> }
>
> @Override
> public DelegatingCollector getAnalyticsCollector(ResponseBuilder rb,
> IndexSearcher searcher) {
> return new TedCollector(myvar, rb);
> }
>
> @Override
> public boolean equals(Object o) {
> if (o instanceof TedQuery) {
> TedQuery tq = (TedQuery) o;
> return Objects.equals(this.myvar, tq.myvar);
> }
> return false;
> }
>
> @Override
> public int hashCode() {
> return myvar == null ? 1 : myvar.hashCode();
> }
>
>
> class TedCollector extends DelegatingCollector {
> ResponseBuilder rb;
> int count;
> String myvar;
>
> public TedCollector(String myvar, ResponseBuilder rb) {
> this.rb = rb;
> this.myvar = myvar;
> }
>
> @Override
> public void collect(int doc) throws IOException {
> count++;
> super.collect(doc);
> }
>
> @Override
> public void finish() throws IOException {
> rb.rsp.add("thecountis", String.valueOf(count));
> rb.rsp.add("myvar", myvar);
>
> if (super.delegate instanceof DelegatingCollector) {
> ((DelegatingCollector) 
> super.delegate).finish();
> }
> }
> }
> }
>
> What am I doing wrong? Thanks!
> Ted
> v5.2.1 SolrCloud mode
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/QParsePlugin-not-working-on-sharded-collection-tp4290249.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Re: Solr not working on new environment

[Jarus Bosman]

OK, solved. It seems I had to first create a core, then configure Drupal to
point to the path for that core.

I have to say, this is one of the more helpful lists I have used. Thanks a
lot for your help!



"Getting information off the Internet is like taking a drink from a fire
hydrant." - Mitchell Kapor

 .---.  .-.   .-..-.   .-.,'|"\.---.,--,
/ .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
| | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
| | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
\ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
 )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
(_)   (__)(_) (__)

On Wed, Mar 30, 2016 at 5:51 PM, Erick Erickson 
wrote:

> Whoa! I thought you were going for SolrCloud. If you're not interested in
> SolrCloud, you don't need to know anything about Zookeeper.
>
> So it looks like Solr is running. You say:
>
> bq:  When I try to connect to :8983/solr, I get a timeout.
> Does it sound like firewall issues?
>
> are you talking about Drupal or about a simple browser connection? If
> the former, I'm all out of ideas
> as I know very little about the Drupal integration and/or whether it's
> even possible with a 5.x...
>
> Best,
> Erick
>
> On Wed, Mar 30, 2016 at 2:52 AM, Jarus Bosman  wrote:
> > OK, an update. I managed to remove the example/cloud directories, and
> stop
> > Solr. I changed my startup script to be much simpler (./solr start) and
> now
> > I get this:
> >
> > *[root@ bin]# ./startsolr.sh*
> > *Waiting up to 30 seconds to see Solr running on port 8983 [|]*
> > *Started Solr server on port 8983 (pid=31937). Happy searching!*
> > *
> >
> >  [root@nationalarchives bin]#
> > ./solr status*
> >
> > *Found 1 Solr nodes:*
> >
> > *Solr process 31937 running on port 8983*
> > *{*
> > *  "solr_home":"/opt/solr-5.5.0/server/solr",*
> > *  "version":"5.5.0 2a228b3920a07f930f7afb6a42d0d20e184a943c - mike -
> > 2016-02-16 15:22:52",*
> > *  "startTime":"2016-03-30T09:24:21.445Z",*
> > *  "uptime":"0 days, 0 hours, 3 minutes, 9 seconds",*
> > *  "memory":"62 MB (%12.6) of 490.7 MB"}*
> >
> > I now want to connect to it from my Drupal installation, but I'm getting
> > this: "The Solr server could not be reached. Further data is therefore
> > unavailable." - I realise this is probably not a Solr error, just giving
> > all the information I have. When I try to connect to
> > :8983/solr, I get a timeout. Does it sound like firewall
> issues?
> >
> > Regards,
> > Jarus
> >
> > "Getting information off the Internet is like taking a drink from a fire
> > hydrant." - Mitchell Kapor
> >
> >  .---.  .-.   .-..-.   .-.,'|"\.---.,--,
> > / .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
> > | | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
> > | | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
> > \ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
> >  )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
> > (_)   (__)(_) (__)
> >
> > On Wed, Mar 30, 2016 at 8:50 AM, Jarus Bosman  wrote:
> >
> >> Hi Erick,
> >>
> >> Thanks for the reply. It seems I have not done all my homework yet.
> >>
> >> We used to use Solr 3.6.2 on the old environment (we're using it in
> >> conjunction with Drupal). When I got connectivity problems on the new
> >> server, I decided to rather implement the latest version of Solr
> (5.5.0). I
> >> read the Quick Start documentation and expected it to work first time,
> but
> >> not so (as per my previous email). I will read up a bit on ZooKeeper
> (never
> >> heard of it before - What is it?). Is there a good place to read up on
> >> getting started with ZooKeeper and the latest versions of Solr (apart
> from
> >> what you have replied, of course)?
> >>
> >> Thank you so much for your assistance,
> >> Jarus
> >>
> >>
> >> "Getting information off the Internet is like taking a drink from a fire
> >> hydrant." - Mitchell Kapor
> >>
> >>  .---.  .-.   .-..-.   .-.,'|"\.---.,--,
> >> / .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
> >> | | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
> >> | | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
> >> \ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
> >>  )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
> >> (_)   (__)(_) (__)
> >>
> >> On Wed, Mar 30, 2016 at 6:20 AM, Erick Erickson <
> erickerick...@gmail.com>
> >> wrote:
> >>
> >>> Good to meet you!
> >>>
> >>> It looks like you've tried to start Solr a time or two. When you start
> >>> up the "cloud" example
> >>> it creates
> >>> /opt/solr-5.5.0/example/cloud
> >>> and puts your SolrCloud stuff under there. It also automatically puts
> >>> your configuration
> >>> sets up on Zookeeper. When I get this kind of thing, I usually
> >>>
> >>> > stop Zookeeper (if running externally)
> >>>
> >>> > rm -rf /opt/solr-5.5.0/example/cloud
> >>>
> >>> > 

Re: Solr not working on new environment

[Erick Erickson]

Whoa! I thought you were going for SolrCloud. If you're not interested in
SolrCloud, you don't need to know anything about Zookeeper.

So it looks like Solr is running. You say:

bq:  When I try to connect to :8983/solr, I get a timeout.
Does it sound like firewall issues?

are you talking about Drupal or about a simple browser connection? If
the former, I'm all out of ideas
as I know very little about the Drupal integration and/or whether it's
even possible with a 5.x...

Best,
Erick

On Wed, Mar 30, 2016 at 2:52 AM, Jarus Bosman  wrote:
> OK, an update. I managed to remove the example/cloud directories, and stop
> Solr. I changed my startup script to be much simpler (./solr start) and now
> I get this:
>
> *[root@ bin]# ./startsolr.sh*
> *Waiting up to 30 seconds to see Solr running on port 8983 [|]*
> *Started Solr server on port 8983 (pid=31937). Happy searching!*
> *
>
>  [root@nationalarchives bin]#
> ./solr status*
>
> *Found 1 Solr nodes:*
>
> *Solr process 31937 running on port 8983*
> *{*
> *  "solr_home":"/opt/solr-5.5.0/server/solr",*
> *  "version":"5.5.0 2a228b3920a07f930f7afb6a42d0d20e184a943c - mike -
> 2016-02-16 15:22:52",*
> *  "startTime":"2016-03-30T09:24:21.445Z",*
> *  "uptime":"0 days, 0 hours, 3 minutes, 9 seconds",*
> *  "memory":"62 MB (%12.6) of 490.7 MB"}*
>
> I now want to connect to it from my Drupal installation, but I'm getting
> this: "The Solr server could not be reached. Further data is therefore
> unavailable." - I realise this is probably not a Solr error, just giving
> all the information I have. When I try to connect to
> :8983/solr, I get a timeout. Does it sound like firewall issues?
>
> Regards,
> Jarus
>
> "Getting information off the Internet is like taking a drink from a fire
> hydrant." - Mitchell Kapor
>
>  .---.  .-.   .-..-.   .-.,'|"\.---.,--,
> / .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
> | | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
> | | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
> \ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
>  )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
> (_)   (__)(_) (__)
>
> On Wed, Mar 30, 2016 at 8:50 AM, Jarus Bosman  wrote:
>
>> Hi Erick,
>>
>> Thanks for the reply. It seems I have not done all my homework yet.
>>
>> We used to use Solr 3.6.2 on the old environment (we're using it in
>> conjunction with Drupal). When I got connectivity problems on the new
>> server, I decided to rather implement the latest version of Solr (5.5.0). I
>> read the Quick Start documentation and expected it to work first time, but
>> not so (as per my previous email). I will read up a bit on ZooKeeper (never
>> heard of it before - What is it?). Is there a good place to read up on
>> getting started with ZooKeeper and the latest versions of Solr (apart from
>> what you have replied, of course)?
>>
>> Thank you so much for your assistance,
>> Jarus
>>
>>
>> "Getting information off the Internet is like taking a drink from a fire
>> hydrant." - Mitchell Kapor
>>
>>  .---.  .-.   .-..-.   .-.,'|"\.---.,--,
>> / .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
>> | | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
>> | | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
>> \ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
>>  )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
>> (_)   (__)(_) (__)
>>
>> On Wed, Mar 30, 2016 at 6:20 AM, Erick Erickson 
>> wrote:
>>
>>> Good to meet you!
>>>
>>> It looks like you've tried to start Solr a time or two. When you start
>>> up the "cloud" example
>>> it creates
>>> /opt/solr-5.5.0/example/cloud
>>> and puts your SolrCloud stuff under there. It also automatically puts
>>> your configuration
>>> sets up on Zookeeper. When I get this kind of thing, I usually
>>>
>>> > stop Zookeeper (if running externally)
>>>
>>> > rm -rf /opt/solr-5.5.0/example/cloud
>>>
>>> > delete all the Zookeeper data. It may take a bit of poking to find out
>>> where
>>> the Zookeeper data is. It's usually in /tmp/zookeeper if you're running ZK
>>> standalone, or in a subdirectory in Solr if you're using embedded ZK.
>>> NOTE: if you're running standalone zookeeper, you should _definitely_
>>> change the data dir because it may disappear from /tmp/zookeeper One
>>> of Zookeeper's little quirks
>>>
>>> > try it all over again.
>>>
>>> Here's the problem. The examples (-e cloud) tries to do a bunch of stuff
>>> for
>>> you to get the installation up and running without having to wend your way
>>> through all of the indiviual commands. Sometimes getting partway through
>>> leaves you in an ambiguous state. Or at least a state you don't quite know
>>> what all the moving parts are.
>>>
>>> Here's the steps you need to follow if you're doing them yourself rather
>>> than
>>> relying on the canned example
>>> 1> start Zookeeper externally. For 

Re: Solr not working on new environment

[Jarus Bosman]

OK, an update. I managed to remove the example/cloud directories, and stop
Solr. I changed my startup script to be much simpler (./solr start) and now
I get this:

*[root@ bin]# ./startsolr.sh*
*Waiting up to 30 seconds to see Solr running on port 8983 [|]*
*Started Solr server on port 8983 (pid=31937). Happy searching!*
*

 [root@nationalarchives bin]#
./solr status*

*Found 1 Solr nodes:*

*Solr process 31937 running on port 8983*
*{*
*  "solr_home":"/opt/solr-5.5.0/server/solr",*
*  "version":"5.5.0 2a228b3920a07f930f7afb6a42d0d20e184a943c - mike -
2016-02-16 15:22:52",*
*  "startTime":"2016-03-30T09:24:21.445Z",*
*  "uptime":"0 days, 0 hours, 3 minutes, 9 seconds",*
*  "memory":"62 MB (%12.6) of 490.7 MB"}*

I now want to connect to it from my Drupal installation, but I'm getting
this: "The Solr server could not be reached. Further data is therefore
unavailable." - I realise this is probably not a Solr error, just giving
all the information I have. When I try to connect to
:8983/solr, I get a timeout. Does it sound like firewall issues?

Regards,
Jarus

"Getting information off the Internet is like taking a drink from a fire
hydrant." - Mitchell Kapor

 .---.  .-.   .-..-.   .-.,'|"\.---.,--,
/ .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
| | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
| | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
\ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
 )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
(_)   (__)(_) (__)

On Wed, Mar 30, 2016 at 8:50 AM, Jarus Bosman  wrote:

> Hi Erick,
>
> Thanks for the reply. It seems I have not done all my homework yet.
>
> We used to use Solr 3.6.2 on the old environment (we're using it in
> conjunction with Drupal). When I got connectivity problems on the new
> server, I decided to rather implement the latest version of Solr (5.5.0). I
> read the Quick Start documentation and expected it to work first time, but
> not so (as per my previous email). I will read up a bit on ZooKeeper (never
> heard of it before - What is it?). Is there a good place to read up on
> getting started with ZooKeeper and the latest versions of Solr (apart from
> what you have replied, of course)?
>
> Thank you so much for your assistance,
> Jarus
>
>
> "Getting information off the Internet is like taking a drink from a fire
> hydrant." - Mitchell Kapor
>
>  .---.  .-.   .-..-.   .-.,'|"\.---.,--,
> / .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
> | | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
> | | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
> \ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
>  )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
> (_)   (__)(_) (__)
>
> On Wed, Mar 30, 2016 at 6:20 AM, Erick Erickson 
> wrote:
>
>> Good to meet you!
>>
>> It looks like you've tried to start Solr a time or two. When you start
>> up the "cloud" example
>> it creates
>> /opt/solr-5.5.0/example/cloud
>> and puts your SolrCloud stuff under there. It also automatically puts
>> your configuration
>> sets up on Zookeeper. When I get this kind of thing, I usually
>>
>> > stop Zookeeper (if running externally)
>>
>> > rm -rf /opt/solr-5.5.0/example/cloud
>>
>> > delete all the Zookeeper data. It may take a bit of poking to find out
>> where
>> the Zookeeper data is. It's usually in /tmp/zookeeper if you're running ZK
>> standalone, or in a subdirectory in Solr if you're using embedded ZK.
>> NOTE: if you're running standalone zookeeper, you should _definitely_
>> change the data dir because it may disappear from /tmp/zookeeper One
>> of Zookeeper's little quirks
>>
>> > try it all over again.
>>
>> Here's the problem. The examples (-e cloud) tries to do a bunch of stuff
>> for
>> you to get the installation up and running without having to wend your way
>> through all of the indiviual commands. Sometimes getting partway through
>> leaves you in an ambiguous state. Or at least a state you don't quite know
>> what all the moving parts are.
>>
>> Here's the steps you need to follow if you're doing them yourself rather
>> than
>> relying on the canned example
>> 1> start Zookeeper externally. For experimentation, a single ZK is quite
>> sufficient, I don't bother with 3 ZK instances and a quorum unless I'm
>> in a production situation.
>> 2> start solr with the bin/solr script, use the -c and -z options. At
>> this point,
>> you have a functioning Solr, but no collections. You should be
>> able to see the solr admin UI at http://node:8982/solr at this point.
>> 3> use the bin/solr zk -upconfig command to put a configset in ZK
>> 4> use the Collections API to create and maintain collections.
>>
>> And one more note. When you use the '-e cloud' option, you'll see
>> messages go by about starting nodes with a command like:
>>
>> bin/solr start -c -z localhost:2181 -p 8981 -s 

Re: Solr not working on new environment

[Jarus Bosman]

Hi Erick,

Thanks for the reply. It seems I have not done all my homework yet.

We used to use Solr 3.6.2 on the old environment (we're using it in
conjunction with Drupal). When I got connectivity problems on the new
server, I decided to rather implement the latest version of Solr (5.5.0). I
read the Quick Start documentation and expected it to work first time, but
not so (as per my previous email). I will read up a bit on ZooKeeper (never
heard of it before - What is it?). Is there a good place to read up on
getting started with ZooKeeper and the latest versions of Solr (apart from
what you have replied, of course)?

Thank you so much for your assistance,
Jarus


"Getting information off the Internet is like taking a drink from a fire
hydrant." - Mitchell Kapor

 .---.  .-.   .-..-.   .-.,'|"\.---.,--,
/ .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
| | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
| | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
\ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
 )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
(_)   (__)(_) (__)

On Wed, Mar 30, 2016 at 6:20 AM, Erick Erickson 
wrote:

> Good to meet you!
>
> It looks like you've tried to start Solr a time or two. When you start
> up the "cloud" example
> it creates
> /opt/solr-5.5.0/example/cloud
> and puts your SolrCloud stuff under there. It also automatically puts
> your configuration
> sets up on Zookeeper. When I get this kind of thing, I usually
>
> > stop Zookeeper (if running externally)
>
> > rm -rf /opt/solr-5.5.0/example/cloud
>
> > delete all the Zookeeper data. It may take a bit of poking to find out
> where
> the Zookeeper data is. It's usually in /tmp/zookeeper if you're running ZK
> standalone, or in a subdirectory in Solr if you're using embedded ZK.
> NOTE: if you're running standalone zookeeper, you should _definitely_
> change the data dir because it may disappear from /tmp/zookeeper One
> of Zookeeper's little quirks
>
> > try it all over again.
>
> Here's the problem. The examples (-e cloud) tries to do a bunch of stuff
> for
> you to get the installation up and running without having to wend your way
> through all of the indiviual commands. Sometimes getting partway through
> leaves you in an ambiguous state. Or at least a state you don't quite know
> what all the moving parts are.
>
> Here's the steps you need to follow if you're doing them yourself rather
> than
> relying on the canned example
> 1> start Zookeeper externally. For experimentation, a single ZK is quite
> sufficient, I don't bother with 3 ZK instances and a quorum unless I'm
> in a production situation.
> 2> start solr with the bin/solr script, use the -c and -z options. At
> this point,
> you have a functioning Solr, but no collections. You should be
> able to see the solr admin UI at http://node:8982/solr at this point.
> 3> use the bin/solr zk -upconfig command to put a configset in ZK
> 4> use the Collections API to create and maintain collections.
>
> And one more note. When you use the '-e cloud' option, you'll see
> messages go by about starting nodes with a command like:
>
> bin/solr start -c -z localhost:2181 -p 8981 -s example/cloud/node1/solr
> bin/solr start -c -z localhost:2181 -p 8982 -s example/cloud/node2/solr
>
> Once the structure is created, then you just use these commands to
> restart an existing set of Solr instances on your machine.
>
> Remember I said that the canned examples create ...example/cloud?
> What the canned examples are doing is creating solr instances that are
> independent, but on the same machine in order to get people started. When
> you specify the '-e cloud' option, those directories are created or, as
> you've
> seen, messages are printed that essentially indicate you're running the
> one-time example... more than once.
>
> HTH,
> Erick
>
>
> On Tue, Mar 29, 2016 at 8:06 AM, Jarus Bosman  wrote:
> > Hi,
> >
> > Introductions first (as I was taught): My name is Jarus Bosman, I am a
> > software developer from South Africa, doing development in Java, PHP and
> > Delphi. I have been programming for 19 years and find out more every day
> > that I don't actually know anything about programming ;).
> >
> > My problem:
> >
> > We recently moved our environment to a new server. I've installed 5.5.0
> on
> > the new environment. When I want to start the server, I get the
> following:
> >
> > *Welcome to the SolrCloud example!*
> >
> > *Starting up 2 Solr nodes for your example SolrCloud cluster.*
> >
> > *Solr home directory /opt/solr-5.5.0/example/cloud/node1/solr already
> > exists.*
> > */opt/solr-5.5.0/example/cloud/node2 already exists.*
> >
> > *Starting up Solr on port 8983 using command:*
> > */opt/solr-5.5.0/bin/solr start -cloud -p 8983 -s
> > "/opt/solr-5.5.0/example/cloud/node1/solr"*
> >
> > *Waiting up to 30 seconds to see Solr running on port 8983 [/]  Still not
> > seeing Solr listening on 

Re: Solr not working on new environment

[Jarus Bosman]

Hi Shyam,

Yes, I've stopped and restarted the process a number of times. I get the
same result every time.

J

"Getting information off the Internet is like taking a drink from a fire
hydrant." - Mitchell Kapor

 .---.  .-.   .-..-.   .-.,'|"\.---.,--,
/ .-. )  ) \_/ /  \ \_/ )/| |\ \  / .-. ) .' .'
| | |(_)(_)   /\   (_)| | \ \ | | |(_)|  |  __
| | | |   / _ \ ) (   | |  \ \| | | | \  \ ( _)
\ `-' /  / / ) \| |   /(|`-' /\ `-' /  \  `-) )
 )---'  `-' (_)-'  /(_|  (__)`--'  )---'   )\/
(_)   (__)(_) (__)

On Wed, Mar 30, 2016 at 4:20 AM, Shyam R  wrote:

> Hi Jarus,
>
> Have you tried stopping the solr process and restarting the cluster again?
>
> Thanks
> Shyam
>
> On Tue, Mar 29, 2016 at 8:36 PM, Jarus Bosman  wrote:
>
> > Hi,
> >
> > Introductions first (as I was taught): My name is Jarus Bosman, I am a
> > software developer from South Africa, doing development in Java, PHP and
> > Delphi. I have been programming for 19 years and find out more every day
> > that I don't actually know anything about programming ;).
> >
> > My problem:
> >
> > We recently moved our environment to a new server. I've installed 5.5.0
> on
> > the new environment. When I want to start the server, I get the
> following:
> >
> > *Welcome to the SolrCloud example!*
> >
> > *Starting up 2 Solr nodes for your example SolrCloud cluster.*
> >
> > *Solr home directory /opt/solr-5.5.0/example/cloud/node1/solr already
> > exists.*
> > */opt/solr-5.5.0/example/cloud/node2 already exists.*
> >
> > *Starting up Solr on port 8983 using command:*
> > */opt/solr-5.5.0/bin/solr start -cloud -p 8983 -s
> > "/opt/solr-5.5.0/example/cloud/node1/solr"*
> >
> > *Waiting up to 30 seconds to see Solr running on port 8983 [/]  Still not
> > seeing Solr listening on 8983 after 30 seconds!*
> > *INFO  - 2016-03-29 14:22:14.356; [   ] org.eclipse.jetty.util.log.Log;
> > Logging initialized @463ms*
> > *INFO  - 2016-03-29 14:22:14.717; [   ] org.eclipse.jetty.server.Server;
> > jetty-9.2.13.v20150730*
> > *WARN  - 2016-03-29 14:22:14.752; [   ]
> > org.eclipse.jetty.server.handler.RequestLogHandler; !RequestLog*
> > *INFO  - 2016-03-29 14:22:14.757; [   ]
> > org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment
> monitor
> > [file:/opt/solr-5.5.0/server/contexts/] at interval 0*
> > *INFO  - 2016-03-29 14:22:15.768; [   ]
> > org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for
> > /solr, did not find org.apache.jasper.servlet.JspServlet*
> > *WARN  - 2016-03-29 14:22:15.790; [   ]
> > org.eclipse.jetty.security.ConstraintSecurityHandler;
> > ServletContext@o.e.j.w.WebAppContext
> >
> @7a583307{/solr,file:/opt/solr-5.5.0/server/solr-webapp/webapp/,STARTING}{/opt/solr-5.5.0/server/solr-webapp/webapp}
> > has uncovered http methods for path: /*
> > *INFO  - 2016-03-29 14:22:15.809; [   ]
> > org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init():
> > WebAppClassLoader=1287618844@4cbf811c*
> > *INFO  - 2016-03-29 14:22:15.848; [   ]
> > org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr
> > (NoInitialContextEx)*
> > *INFO  - 2016-03-29 14:22:15.849; [   ]
> > org.apache.solr.core.SolrResourceLoader; using system property
> > solr.solr.home: /opt/solr-5.5.0/example/cloud/node1/solr*
> > *INFO  - 2016-03-29 14:22:15.850; [   ]
> > org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for
> > directory: '/opt/solr-5.5.0/example/cloud/node1/solr'*
> > *INFO  - 2016-03-29 14:22:15.851; [   ]
> > org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr
> > (NoInitialContextEx)*
> > *INFO  - 2016-03-29 14:22:15.852; [   ]
> > org.apache.solr.core.SolrResourceLoader; using system property
> > solr.solr.home: /opt/solr-5.5.0/example/cloud/node1/solr*
> > *INFO  - 2016-03-29 14:22:15.880; [   ]
> org.apache.solr.core.SolrXmlConfig;
> > Loading container configuration from
> > /opt/solr-5.5.0/example/cloud/node1/solr/solr.xml*
> > *INFO  - 2016-03-29 14:22:16.051; [   ]
> > org.apache.solr.core.CorePropertiesLocator; Config-defined core root
> > directory: /opt/solr-5.5.0/example/cloud/node1/solr*
> > *INFO  - 2016-03-29 14:22:16.104; [   ]
> org.apache.solr.core.CoreContainer;
> > New CoreContainer 1211012646*
> > *INFO  - 2016-03-29 14:22:16.104; [   ]
> org.apache.solr.core.CoreContainer;
> > Loading cores into CoreContainer
> > [instanceDir=/opt/solr-5.5.0/example/cloud/node1/solr]*
> > *WARN  - 2016-03-29 14:22:16.109; [   ]
> org.apache.solr.core.CoreContainer;
> > Couldn't add files from /opt/solr-5.5.0/example/cloud/node1/solr/lib to
> > classpath: /opt/solr-5.5.0/example/cloud/node1/solr/lib*
> > *INFO  - 2016-03-29 14:22:16.133; [   ]
> > org.apache.solr.handler.component.HttpShardHandlerFactory; created with
> > socketTimeout : 60,connTimeout : 6,maxConnectionsPerHost :
> > 20,maxConnections : 1,corePoolSize : 0,maximumPoolSize :
> > 

Re: Solr not working on new environment

[Erick Erickson]

Good to meet you!

It looks like you've tried to start Solr a time or two. When you start
up the "cloud" example
it creates
/opt/solr-5.5.0/example/cloud
and puts your SolrCloud stuff under there. It also automatically puts
your configuration
sets up on Zookeeper. When I get this kind of thing, I usually

> stop Zookeeper (if running externally)

> rm -rf /opt/solr-5.5.0/example/cloud

> delete all the Zookeeper data. It may take a bit of poking to find out where
the Zookeeper data is. It's usually in /tmp/zookeeper if you're running ZK
standalone, or in a subdirectory in Solr if you're using embedded ZK.
NOTE: if you're running standalone zookeeper, you should _definitely_
change the data dir because it may disappear from /tmp/zookeeper One
of Zookeeper's little quirks

> try it all over again.

Here's the problem. The examples (-e cloud) tries to do a bunch of stuff for
you to get the installation up and running without having to wend your way
through all of the indiviual commands. Sometimes getting partway through
leaves you in an ambiguous state. Or at least a state you don't quite know
what all the moving parts are.

Here's the steps you need to follow if you're doing them yourself rather than
relying on the canned example
1> start Zookeeper externally. For experimentation, a single ZK is quite
sufficient, I don't bother with 3 ZK instances and a quorum unless I'm
in a production situation.
2> start solr with the bin/solr script, use the -c and -z options. At
this point,
you have a functioning Solr, but no collections. You should be
able to see the solr admin UI at http://node:8982/solr at this point.
3> use the bin/solr zk -upconfig command to put a configset in ZK
4> use the Collections API to create and maintain collections.

And one more note. When you use the '-e cloud' option, you'll see
messages go by about starting nodes with a command like:

bin/solr start -c -z localhost:2181 -p 8981 -s example/cloud/node1/solr
bin/solr start -c -z localhost:2181 -p 8982 -s example/cloud/node2/solr

Once the structure is created, then you just use these commands to
restart an existing set of Solr instances on your machine.

Remember I said that the canned examples create ...example/cloud?
What the canned examples are doing is creating solr instances that are
independent, but on the same machine in order to get people started. When
you specify the '-e cloud' option, those directories are created or, as you've
seen, messages are printed that essentially indicate you're running the
one-time example... more than once.

HTH,
Erick


On Tue, Mar 29, 2016 at 8:06 AM, Jarus Bosman  wrote:
> Hi,
>
> Introductions first (as I was taught): My name is Jarus Bosman, I am a
> software developer from South Africa, doing development in Java, PHP and
> Delphi. I have been programming for 19 years and find out more every day
> that I don't actually know anything about programming ;).
>
> My problem:
>
> We recently moved our environment to a new server. I've installed 5.5.0 on
> the new environment. When I want to start the server, I get the following:
>
> *Welcome to the SolrCloud example!*
>
> *Starting up 2 Solr nodes for your example SolrCloud cluster.*
>
> *Solr home directory /opt/solr-5.5.0/example/cloud/node1/solr already
> exists.*
> */opt/solr-5.5.0/example/cloud/node2 already exists.*
>
> *Starting up Solr on port 8983 using command:*
> */opt/solr-5.5.0/bin/solr start -cloud -p 8983 -s
> "/opt/solr-5.5.0/example/cloud/node1/solr"*
>
> *Waiting up to 30 seconds to see Solr running on port 8983 [/]  Still not
> seeing Solr listening on 8983 after 30 seconds!*
> *INFO  - 2016-03-29 14:22:14.356; [   ] org.eclipse.jetty.util.log.Log;
> Logging initialized @463ms*
> *INFO  - 2016-03-29 14:22:14.717; [   ] org.eclipse.jetty.server.Server;
> jetty-9.2.13.v20150730*
> *WARN  - 2016-03-29 14:22:14.752; [   ]
> org.eclipse.jetty.server.handler.RequestLogHandler; !RequestLog*
> *INFO  - 2016-03-29 14:22:14.757; [   ]
> org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor
> [file:/opt/solr-5.5.0/server/contexts/] at interval 0*
> *INFO  - 2016-03-29 14:22:15.768; [   ]
> org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for
> /solr, did not find org.apache.jasper.servlet.JspServlet*
> *WARN  - 2016-03-29 14:22:15.790; [   ]
> org.eclipse.jetty.security.ConstraintSecurityHandler;
> ServletContext@o.e.j.w.WebAppContext@7a583307{/solr,file:/opt/solr-5.5.0/server/solr-webapp/webapp/,STARTING}{/opt/solr-5.5.0/server/solr-webapp/webapp}
> has uncovered http methods for path: /*
> *INFO  - 2016-03-29 14:22:15.809; [   ]
> org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init():
> WebAppClassLoader=1287618844@4cbf811c*
> *INFO  - 2016-03-29 14:22:15.848; [   ]
> org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr
> (NoInitialContextEx)*
> *INFO  - 2016-03-29 14:22:15.849; [   ]
> org.apache.solr.core.SolrResourceLoader; using system 

Re: Solr not working on new environment

[Shyam R]

Hi Jarus,

Have you tried stopping the solr process and restarting the cluster again?

Thanks
Shyam

On Tue, Mar 29, 2016 at 8:36 PM, Jarus Bosman  wrote:

> Hi,
>
> Introductions first (as I was taught): My name is Jarus Bosman, I am a
> software developer from South Africa, doing development in Java, PHP and
> Delphi. I have been programming for 19 years and find out more every day
> that I don't actually know anything about programming ;).
>
> My problem:
>
> We recently moved our environment to a new server. I've installed 5.5.0 on
> the new environment. When I want to start the server, I get the following:
>
> *Welcome to the SolrCloud example!*
>
> *Starting up 2 Solr nodes for your example SolrCloud cluster.*
>
> *Solr home directory /opt/solr-5.5.0/example/cloud/node1/solr already
> exists.*
> */opt/solr-5.5.0/example/cloud/node2 already exists.*
>
> *Starting up Solr on port 8983 using command:*
> */opt/solr-5.5.0/bin/solr start -cloud -p 8983 -s
> "/opt/solr-5.5.0/example/cloud/node1/solr"*
>
> *Waiting up to 30 seconds to see Solr running on port 8983 [/]  Still not
> seeing Solr listening on 8983 after 30 seconds!*
> *INFO  - 2016-03-29 14:22:14.356; [   ] org.eclipse.jetty.util.log.Log;
> Logging initialized @463ms*
> *INFO  - 2016-03-29 14:22:14.717; [   ] org.eclipse.jetty.server.Server;
> jetty-9.2.13.v20150730*
> *WARN  - 2016-03-29 14:22:14.752; [   ]
> org.eclipse.jetty.server.handler.RequestLogHandler; !RequestLog*
> *INFO  - 2016-03-29 14:22:14.757; [   ]
> org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor
> [file:/opt/solr-5.5.0/server/contexts/] at interval 0*
> *INFO  - 2016-03-29 14:22:15.768; [   ]
> org.eclipse.jetty.webapp.StandardDescriptorProcessor; NO JSP Support for
> /solr, did not find org.apache.jasper.servlet.JspServlet*
> *WARN  - 2016-03-29 14:22:15.790; [   ]
> org.eclipse.jetty.security.ConstraintSecurityHandler;
> ServletContext@o.e.j.w.WebAppContext
> @7a583307{/solr,file:/opt/solr-5.5.0/server/solr-webapp/webapp/,STARTING}{/opt/solr-5.5.0/server/solr-webapp/webapp}
> has uncovered http methods for path: /*
> *INFO  - 2016-03-29 14:22:15.809; [   ]
> org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init():
> WebAppClassLoader=1287618844@4cbf811c*
> *INFO  - 2016-03-29 14:22:15.848; [   ]
> org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr
> (NoInitialContextEx)*
> *INFO  - 2016-03-29 14:22:15.849; [   ]
> org.apache.solr.core.SolrResourceLoader; using system property
> solr.solr.home: /opt/solr-5.5.0/example/cloud/node1/solr*
> *INFO  - 2016-03-29 14:22:15.850; [   ]
> org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for
> directory: '/opt/solr-5.5.0/example/cloud/node1/solr'*
> *INFO  - 2016-03-29 14:22:15.851; [   ]
> org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr
> (NoInitialContextEx)*
> *INFO  - 2016-03-29 14:22:15.852; [   ]
> org.apache.solr.core.SolrResourceLoader; using system property
> solr.solr.home: /opt/solr-5.5.0/example/cloud/node1/solr*
> *INFO  - 2016-03-29 14:22:15.880; [   ] org.apache.solr.core.SolrXmlConfig;
> Loading container configuration from
> /opt/solr-5.5.0/example/cloud/node1/solr/solr.xml*
> *INFO  - 2016-03-29 14:22:16.051; [   ]
> org.apache.solr.core.CorePropertiesLocator; Config-defined core root
> directory: /opt/solr-5.5.0/example/cloud/node1/solr*
> *INFO  - 2016-03-29 14:22:16.104; [   ] org.apache.solr.core.CoreContainer;
> New CoreContainer 1211012646*
> *INFO  - 2016-03-29 14:22:16.104; [   ] org.apache.solr.core.CoreContainer;
> Loading cores into CoreContainer
> [instanceDir=/opt/solr-5.5.0/example/cloud/node1/solr]*
> *WARN  - 2016-03-29 14:22:16.109; [   ] org.apache.solr.core.CoreContainer;
> Couldn't add files from /opt/solr-5.5.0/example/cloud/node1/solr/lib to
> classpath: /opt/solr-5.5.0/example/cloud/node1/solr/lib*
> *INFO  - 2016-03-29 14:22:16.133; [   ]
> org.apache.solr.handler.component.HttpShardHandlerFactory; created with
> socketTimeout : 60,connTimeout : 6,maxConnectionsPerHost :
> 20,maxConnections : 1,corePoolSize : 0,maximumPoolSize :
> 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy :
> false,useRetries : false,*
> *INFO  - 2016-03-29 14:22:16.584; [   ]
> org.apache.solr.update.UpdateShardHandler; Creating UpdateShardHandler HTTP
> client with params: socketTimeout=60=6=true*
> *INFO  - 2016-03-29 14:22:16.590; [   ] org.apache.solr.logging.LogWatcher;
> SLF4J impl is org.slf4j.impl.Log4jLoggerFactory*
> *INFO  - 2016-03-29 14:22:16.592; [   ] org.apache.solr.logging.LogWatcher;
> Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]*
> *INFO  - 2016-03-29 14:22:16.603; [   ]
> org.apache.solr.cloud.SolrZkServerProps; Reading configuration from:
> /opt/solr-5.5.0/example/cloud/node1/solr/zoo.cfg*
> *INFO  - 2016-03-29 14:22:16.605; [   ] org.apache.solr.cloud.SolrZkServer;
> STARTING EMBEDDED STANDALONE ZOOKEEPER SERVER at port 9983*

Re: Solr stops working...randomly

[Shawn Heisey]

On 2/24/2016 11:19 AM, Michael Beccaria wrote:
> We're running solr 4.4.0 running in this software 
> (https://github.com/CDRH/nebnews - Django based newspaper site). Solr is 
> running on Ubuntu 12.04 in Jetty. The site occasionally (once a day) goes 
> down with a Connection Refused error. I’m having a hard time troubleshooting 
> the issue and was looking for help in next steps in trying to find out why it 
> is failing.
>
> After debugging it turns out that it is solr that is refusing the connection 
> (restarting Jetty fixes it every time). It randomly fails.

The immediate possibility for the cause of this problem that comes to
mind is the maxThreads parameter in Jetty.  Beyond that, there is also
the OS process limit.

The maxThreads parameter in the Jetty config defaults to 200, and it is
quite easy to exceed this.  In the Jetty that comes packaged with Solr,
this setting has been changed to 1, which effectively removes the
limit for a typical Solr install.  Because you are running 4.4 and your
message indicates you are using "service jetty" commands, chances are
that you are NOT using the jetty that came with Solr.  The first thing I
would try is increasing the maxThreads parameter to 1.

The process limit is increased in /etc/security/limits.conf.  Here are
the additions that I make to this file on my Solr servers, to increase
the limits on the number of processes/threads and open files, both of
which default to 1024:

solrhardnproc   6144
solrsoftnproc   4096

solrhardnofile  65535
solrsoftnofile  49151

Thanks,
Shawn

Re: solrcloud shard working mechanism

[Binoy Dalal]

Check out this Sharding wiki page:
https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud

It has a pretty comprehensive overview of how Sharding works.

On Wed, 24 Feb 2016, 00:46 Mugeesh Husain  wrote:

> Hello,
>
> could some explain about shard system, if we created 10 shard in a
> collection then search
>
> localhost:8983/solr/collection?q=id:12... in which shard this id will be
> search first.
>
> I mean i need to know internal working standard of shard/searching in
> solrlcoud.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/solrcloud-shard-working-mechanism-tp4259261.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
-- 
Regards,
Binoy Dalal

Re: EdgeNGramFilterFactory not working? Solr 5.3.1

[Alexandre Rafalovitch]

Here would be my debugging sequence:

1. Are you actually searching against: dispNamePrefix (and not against
the default text field which has its own analyzer stack)?
2. Do you see the field definition in the Schema Browser screen?
3. If you on that screen, click "Load Term Info" do you see the partial terms?
4. If you go to the Analysis screen, you should be able to select the
field (or the type) from the drop-down and put both text to index and
text to search and see what happens to them and whether they match.

Regards,
   Alex.

Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 17 November 2015 at 18:17, Daniel Valdivia  wrote:
> Hi,
>
> I'm trying to get the EdgeNGramFilterFactory filter to work on a certain 
> field, however after defining the fieldType, creating a field for it and 
> copying the source, this doesn't seem to be working.
>
> One catch here, that I'm not sure if it's affecting the outcome is that none 
> of my fields are stored, everything but the document id in my index is 
> stored=false
>
> I'm using Solr 5.3.1, and I know in my corpus the word "incident" is present, 
> I can search for it, but looking for "inci" yields no results
>
> http://localhost:8983/solr/superCore/select?q=inci=record_display_name=json=true
>
> Any idea on what could I be doing wrong?
>
> This is how I define the field type
>
> {
>   "add-field-type" : {
> "indexed" : true,
> "queryAnalyzer" : {
>   "filters" : [
> {
>   "class" : "solr.LowerCaseFilterFactory"
> }
>   ],
>   "tokenizer" : {
> "class" : "solr.WhitespaceTokenizerFactory"
>   }
> },
> "indexAnalyzer" : {
>   "filters" : [
> {
>   "class" : "solr.LowerCaseFilterFactory"
> },
> {
>   "class" : "solr.EdgeNGramFilterFactory",
>   "minGramSize" : "2",
>   "maxGramSize" : "10"
> }
>   ],
>   "tokenizer" : {
> "class" : "solr.WhitespaceTokenizerFactory"
>   }
> },
> "stored" : false,
> "name" : "prefix",
> "class" : "solr.TextField"
>   }
> }
>
> Adding the field
>
> {
>   "add-field":{
>  "name":"dispNamePrefix",
>  "type":"prefix",
>  "stored":false }
> }
>
> Copy field
>
> {
>   "add-copy-field":{
>  "source":"record_display_name",
>  "dest":[ "dispNamePrefix"]}
> }

Re: EdgeNGramFilterFactory not working? Solr 5.3.1

[Daniel Valdivia]

Hi Markus,

I did, everytime I run this experiment I start from 0 :)

However, after the last change I did seems like I forgot to commit and I 
couldn't get results, so now I have some results.

The resolution to this problem was specifying the search in the dispNamePrefix 
field :O

Thanks Markus and Alexandre

> On Nov 17, 2015, at 3:40 PM, Markus Jelsma  wrote:
> 
> Hi - the usual suspect is: 'did you reindex?' Not seeing things change after 
> modifying index-time analysis chains means you need to reindex.
> 
> M.
> 
> 
> 
> -Original message-
>> From:Daniel Valdivia 
>> Sent: Wednesday 18th November 2015 0:17
>> To: solr-user@lucene.apache.org
>> Subject: EdgeNGramFilterFactory not working? Solr 5.3.1
>> 
>> Hi,
>> 
>> I'm trying to get the EdgeNGramFilterFactory filter to work on a certain 
>> field, however after defining the fieldType, creating a field for it and 
>> copying the source, this doesn't seem to be working.
>> 
>> One catch here, that I'm not sure if it's affecting the outcome is that none 
>> of my fields are stored, everything but the document id in my index is 
>> stored=false
>> 
>> I'm using Solr 5.3.1, and I know in my corpus the word "incident" is 
>> present, I can search for it, but looking for "inci" yields no results
>> 
>> http://localhost:8983/solr/superCore/select?q=inci=record_display_name=json=true
>> 
>> Any idea on what could I be doing wrong?
>> 
>> This is how I define the field type
>> 
>> {
>>  "add-field-type" : {
>>"indexed" : true,
>>"queryAnalyzer" : {
>>  "filters" : [
>>{
>>  "class" : "solr.LowerCaseFilterFactory"
>>}
>>  ],
>>  "tokenizer" : {
>>"class" : "solr.WhitespaceTokenizerFactory"
>>  }
>>},
>>"indexAnalyzer" : {
>>  "filters" : [
>>{
>>  "class" : "solr.LowerCaseFilterFactory"
>>},
>>{
>>  "class" : "solr.EdgeNGramFilterFactory",
>>  "minGramSize" : "2",
>>  "maxGramSize" : "10"
>>}
>>  ],
>>  "tokenizer" : {
>>"class" : "solr.WhitespaceTokenizerFactory"
>>  }
>>},
>>"stored" : false,
>>"name" : "prefix",
>>"class" : "solr.TextField"
>>  }
>> }
>> 
>> Adding the field
>> 
>> {
>>  "add-field":{
>> "name":"dispNamePrefix",
>> "type":"prefix",
>> "stored":false }
>> }
>> 
>> Copy field
>> 
>> {
>>  "add-copy-field":{
>> "source":"record_display_name",
>> "dest":[ "dispNamePrefix"]}
>> }

RE: EdgeNGramFilterFactory not working? Solr 5.3.1

[Markus Jelsma]

Hi - the usual suspect is: 'did you reindex?' Not seeing things change after 
modifying index-time analysis chains means you need to reindex.

M.

 
 
-Original message-
> From:Daniel Valdivia 
> Sent: Wednesday 18th November 2015 0:17
> To: solr-user@lucene.apache.org
> Subject: EdgeNGramFilterFactory not working? Solr 5.3.1
> 
> Hi,
> 
> I'm trying to get the EdgeNGramFilterFactory filter to work on a certain 
> field, however after defining the fieldType, creating a field for it and 
> copying the source, this doesn't seem to be working.
> 
> One catch here, that I'm not sure if it's affecting the outcome is that none 
> of my fields are stored, everything but the document id in my index is 
> stored=false
> 
> I'm using Solr 5.3.1, and I know in my corpus the word "incident" is present, 
> I can search for it, but looking for "inci" yields no results
> 
> http://localhost:8983/solr/superCore/select?q=inci=record_display_name=json=true
> 
> Any idea on what could I be doing wrong?
> 
> This is how I define the field type
> 
> {
>   "add-field-type" : {
> "indexed" : true,
> "queryAnalyzer" : {
>   "filters" : [
> {
>   "class" : "solr.LowerCaseFilterFactory"
> }
>   ],
>   "tokenizer" : {
> "class" : "solr.WhitespaceTokenizerFactory"
>   }
> },
> "indexAnalyzer" : {
>   "filters" : [
> {
>   "class" : "solr.LowerCaseFilterFactory"
> },
> {
>   "class" : "solr.EdgeNGramFilterFactory",
>   "minGramSize" : "2",
>   "maxGramSize" : "10"
> }
>   ],
>   "tokenizer" : {
> "class" : "solr.WhitespaceTokenizerFactory"
>   }
> },
> "stored" : false,
> "name" : "prefix",
> "class" : "solr.TextField"
>   }
> }
> 
> Adding the field
> 
> {
>   "add-field":{
>  "name":"dispNamePrefix",
>  "type":"prefix",
>  "stored":false }
> }
> 
> Copy field
> 
> {
>   "add-copy-field":{
>  "source":"record_display_name",
>  "dest":[ "dispNamePrefix"]}
> }