Re: Location of Solr 9 Branch

[Houston Putman]

Solr 9 is an unreleased major version, so it lives in *master*. Once the
release process starts for Solr 9, it will live at *branch_9x*, and *master*
will host Solr 10.

On Tue, Mar 2, 2021 at 3:49 PM Phill Campbell 
wrote:

> I have just begun investigating Solr source code. Where is the branch for
> Solr 9?
>
>
>

Re: Solr Slack Workspace

[Houston Putman]

There is https://solr-dev.slack.com

It's not really used, but it's there and we can open it up for people to
join and start using.

On Tue, Jan 26, 2021 at 5:38 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Thanks ufuk. I'll take a look.
>
> On Tue, 26 Jan, 2021, 4:05 pm ufuk yılmaz, 
> wrote:
>
> > It’s asking for a searchscale.com email address?
> >
> > Sent from Mail for Windows 10
> >
> > From: Ishan Chattopadhyaya
> > Sent: 26 January 2021 13:33
> > To: solr-user
> > Subject: Re: Solr Slack Workspace
> >
> > There is a Slack backed by official IRC support. Please see
> > https://lucene.472066.n3.nabble.com/Solr-Users-Slack-td4466856.html for
> > details on how to join it.
> >
> > On Tue, 19 Jan, 2021, 2:54 pm Charlie Hull, <
> > ch...@opensourceconnections.com>
> > wrote:
> >
> > > Relevance Slack is open to anyone working on search & relevance - #solr
> > is
> > > only one of the channels, there's lots more! Hope to see you there.
> > >
> > > Cheers
> > >
> > > Charlie
> > > https://opensourceconnections.com/slack
> > >
> > >
> > > On 16/01/2021 02:18, matthew sporleder wrote:
> > > > IRC has kind of died off,
> > > > https://lucene.apache.org/solr/community.html has a slack mentioned,
> > > > I'm on https://opensourceconnections.com/slack after taking their
> solr
> > > > training class and assume it's mostly open to solr community.
> > > >
> > > > On Fri, Jan 15, 2021 at 8:10 PM Justin Sweeney
> > > >  wrote:
> > > >> Hi all,
> > > >>
> > > >> I did some googling and didn't find anything, but is there a Slack
> > > >> workspace for Solr? I think this could be useful to expand
> interaction
> > > >> within the community of Solr users and connect people solving
> similar
> > > >> problems.
> > > >>
> > > >> I'd be happy to get this setup if it does not exist already.
> > > >>
> > > >> Justin
> > >
> > >
> > > --
> > > Charlie Hull - Managing Consultant at OpenSource Connections Limited
> > > 
> > > Founding member of The Search Network 
> > > and co-author of Searching the Enterprise
> > > 
> > > tel/fax: +44 (0)8700 118334
> > > mobile: +44 (0)7767 825828
> > >
> >
> >
>

Re: Solr cloud issuse: Async exception during distributed update

[Houston Putman]

Do you have a field named "314257s_seourls" in your schema?

Is there a dynamic field you are trying to match with that name?

- Houston

On Thu, Dec 10, 2020 at 2:53 PM ritvik  wrote:

> Hi ,
>  Please suggest, why it is happening.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: solrcloud with EKS kubernetes

[Houston Putman]

Hello Abhishek,

It's really hard to provide any advice without knowing any information
about your setup/usage.

Are you giving your Solr pods enough resources on EKS?
Have you run Solr in the same configuration outside of kubernetes in the
past without timeouts?
What type of storage volumes are you using to store your data?
Are you using headless services to connect your Solr Nodes, or ingresses?

If this is the first time that you are using this data + Solr
configuration, maybe it's just that your data within Solr isn't optimized
for the type of queries that you are doing.
If you have run it successfully in the past outside of Kubernetes, then I
would look at the resources that you are giving your pods and the storage
volumes that you are using.
If you are using Ingresses, that might be causing slow connections between
nodes, or between your client and Solr.

- Houston

On Wed, Dec 9, 2020 at 3:24 PM Abhishek Mishra  wrote:

> Hello guys,
> We are kind of facing some of the issues(Like timeout etc.) which are very
> inconsistent. By any chance can it be related to EKS? We are using solr 7.7
> and zookeeper 3.4.13. Should we move to ECS?
>
> Regards,
> Abhishek
>

Re: Unable to upload updated solr config set

[Houston Putman]

Until the next Solr minor version is released you will not be able to
overwrite an existing configSet with a new configSet of the same name.

The ticket for this feature is SOLR-10391
, and it will be included
in the 8.7.0 release.

Until then you will have to create a configSet with a new name, and then
update your collections to point to that new configSet.

- Houston

On Sun, Sep 27, 2020 at 6:56 PM Deepu  wrote:

> Hi,
>
> I was able to upload solr  configs using solr/admin/configs?action=UPLOAD
> api but getting below error when reupload same config set with same.
>
> {
>
>   "responseHeader":{
>
> "status":400,
>
> "QTime":51},
>
>   "error":{
>
> "metadata":[
>
>   "error-class","org.apache.solr.common.SolrException",
>
>   "root-error-class","org.apache.solr.common.SolrException"],
>
> "msg":"The configuration sampleConfigSet already exists in zookeeper",
>
> "code":400}}
>
>
> how we re upload same config with few schema & solr config changes ?
>
>
>
> Thanks,
>
> Deepu
>

Re: Updating configset

[Houston Putman]

I completely agree, there should be a way to overwrite an existing
configSet.

Looks like https://issues.apache.org/jira/browse/SOLR-10391 already exists,
so the work could be tracked there.

On Fri, Sep 11, 2020 at 12:36 PM Tomás Fernández Löbbe <
tomasflo...@gmail.com> wrote:

> I was in the same situation recently. I think it would be nice to have the
> configset UPLOAD command be able to override the existing configset instead
> of just fail (with a parameter such as override=true or something). We need
> to be careful with the trusted/unstrusted flag there, but that should be
> possible.
>
> > If we can’t modify the configset wholesale this way, is it possible to
> create a new configset and swap the old collection to it?
> You can create a new one and then call MODIFYCOLLECTION on the collection
> that uses it:
>
> https://lucene.apache.org/solr/guide/8_6/collection-management.html#modifycollection-parameters
> .
> I've never used that though.
>
> On Fri, Sep 11, 2020 at 7:26 AM Carroll, Michael (ELS-PHI) <
> m.carr...@elsevier.com> wrote:
>
> > Hello,
> >
> > I am running SolrCloud in Kubernetes with Solr version 8.5.2.
> >
> > Is it possible to update a configset being used by a collection using a
> > SolrCloud API directly? I know that this is possible using the zkcli and
> a
> > collection RELOAD. We essentially want to be able to checkout our
> configset
> > from source control, and then replace everything in the active configset
> in
> > SolrCloud (other than the schema.xml).
> >
> > We have a couple of custom plugins that use config files that reside in
> > the configset, and we don’t want to have to rebuild the collection or
> > access zookeeper directly if we don’t have to. If we can’t modify the
> > configset wholesale this way, is it possible to create a new configset
> and
> > swap the old collection to it?
> >
> > Best,
> > Michael Carroll
> >
>

Re: How to Prevent Recovery?

[Houston Putman]

Are you able to use TLOG replicas? That should reduce the time it takes to
recover significantly. It doesn't seem like you have a hard need for
near-real-time, since slow ingestions are fine.

- Houston

On Tue, Aug 25, 2020 at 12:03 PM Anshuman Singh 
wrote:

> Hi,
>
> We have a 10 node (150G RAM, 1TB SAS HDD, 32 cores) Solr 8.5.1 cluster with
> 50 shards, rf 2 (NRT replicas), 7B docs, We have 5 Zk with 2 running on the
> same nodes where Solr is running. Our use case requires continuous
> ingestions (updates mostly). If we ingest at 40k records per sec, after
> 10-15mins some replicas go into recovery with the errors observed given in
> the end. We also observed high CPU during these ingestions (60-70%) and
> disks frequently reach 100% utilization.
>
> We know our hardware is limited but this system will be used by only a few
> users and search times taking a few minutes and slow ingestions are fine so
> we are trying to run with these specifications for now but recovery is
> becoming a bottleneck.
>
> So to prevent recovery which I'm thinking could be due to high CPU/Disk
> during ingestions, we reduced the data rate to 10k records per sec. Now CPU
> usage is not high and recovery is not that frequent but it can happen in a
> long run of 2-3 hrs. We further reduced the rate to 4k records per sec but
> again it happened after 3-4 hrs. Logs were filled with the below error on
> the instance on which recovery happened. Seems like reducing data rate is
> not helping with recovery.
>
> *2020-08-25 12:16:11.008 ERROR (qtp1546693040-235) [c:collection s:shard41
> r:core_node565 x:collection_shard41_replica_n562] o.a.s.s.HttpSolrCall
> null:java.io.IOException: java.util.concurrent.TimeoutException: Idle
> timeout expired: 30/30 ms*
>
> Solr thread dump showed commit threads taking upto 10-15 minutes. Currently
> auto commit happens at 10M docs or 30seconds.
>
> Can someone point me in the right direction? Also can we perform
> core-binding for Solr processes?
>
> *2020-08-24 12:32:55.835 WARN  (zkConnectionManagerCallback-11-thread-1) [
>   ] o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@372ea2bc name:
> ZooKeeperConnection Watcher:x.x.x.7:2181,x.x.x.8:2181/solr got event
> WatchedEvent state:Disconnected type:None path:null path: null type: None*
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2020-08-24 12:41:02.005 WARN  (main-SendThread(x.x.x.8:2181)) [   ]
> o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session
> 0x273f9a8fb229269 has expired2020-08-24 12:41:06.177 WARN
>  (MetricsHistoryHandler-8-thread-1) [   ] o.a.s.h.a.MetricsHistoryHandler
> Could not obtain overseer's address, skipping. =>
> org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leaderat
>
> org.apache.zookeeper.KeeperException.create(KeeperException.java:134)org.apache.zookeeper.KeeperException$SessionExpiredException:
> KeeperErrorCode = Session expired for /overseer_elect/leaderat
> org.apache.zookeeper.KeeperException.create(KeeperException.java:134)
> ~[?:?]at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
> ~[?:?]at
> org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:2131)
> ~[?:?]2020-08-24 12:41:13.365 WARN
>  (zkConnectionManagerCallback-11-thread-1) [   ]
> o.a.s.c.c.ConnectionManager Watcher
> org.apache.solr.common.cloud.ConnectionManager@372ea2bc name:
> ZooKeeperConnection Watcher:x.x.x.7:2181,x.x.x.8:2181/solr got event
> WatchedEvent state:Expired type:None path:null path: null type:
> None2020-08-24 12:41:13.366 WARN  (zkConnectionManagerCallback-11-thread-1)
> [   ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was
> expired. Attempting to reconnect to recover relationship with
> ZooKeeper...2020-08-24 12:41:16.705 ERROR (qtp1546693040-163255)
> [c:collection s:shard31 r:core_node525 x:collection_shard31_replica_n522]
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Cannot
> talk to ZooKeeper - Updates are disabled*
>

[ANNOUNCE] Apache Solr 8.6.1 released

[Houston Putman]

The Lucene PMC is pleased to announce the release of Apache Solr 8.6.1.

Solr is the popular, blazing fast, open source NoSQL search platform from
the Apache Lucene project. Its major features include powerful full-text
search, hit highlighting, faceted search, dynamic clustering, database
integration, rich document handling, and geospatial search. Solr is highly
scalable, providing fault tolerant distributed search and indexing, and
powers the search and navigation features of many of the world's largest
internet sites.

Solr 8.6.1 is available for immediate download at:

  

### Solr 8.6.1 Release Highlights:

 * SOLR-14665: Revert SOLR-12845 adding of default autoscaling cluster
policy, due to performance issues
 * SOLR-14671: Parsing dynamic ZK config sometimes cause
NumberFormatException

Please refer to the Upgrade Notes in the Solr Ref Guide for information on
upgrading from previous Solr versions:

  

Please read CHANGES.txt for a full list of bugfixes:

  

Solr 8.6.1 also includes bugfixes in the corresponding Apache Lucene
release:

  

Note: The Apache Software Foundation uses an extensive mirroring network
for distributing releases. It is possible that the mirror you are using may
not have replicated the release yet. If that is the case, please try
another mirror.
This also applies to Maven access.

Re: [ANNOUNCE] Apache Solr 8.6.0 released

[Houston Putman]

To address your concern Bernd,

The goal of this deprecation is not to remove the functionality entirely.
The primary purpose is to remove the code from Solr Core. Before removing a
feature we aim to either:

   - Move the code to another repository, and have it be loadable via a
   plugin
   - Replace the feature with something more stable and/or scalable.
   (Likely loadable via a plugin or run in a separate application)

I understand your frustration, but the ultimate goal here is to make Solr
more customizable and plugable (and therefore learner by default). This way
the base Solr functionality can be as bug-free and performant as possible,
and any extra features can be added on top as needed.

We would appreciate feedback for how the community would prefer these
features be provided in the future, so that we make the transition smoother
and the end product better.

- Houston

On Wed, Jul 15, 2020 at 5:51 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> On Wed, 15 Jul, 2020, 8:37 pm Bernd Fehling, <
> bernd.fehl...@uni-bielefeld.de>
> wrote:
>
> >
> >
> > Am 15.07.20 um 16:07 schrieb Ishan Chattopadhyaya:
> > > Dear Solr Users,
> > >
> > > In this release (Solr 8.6), we have deprecated the following:
> > >
> > >   1. Data Import Handler
> > >
> > >   2. HDFS support
> > >
> > >   3. Cross Data Center Replication (CDCR)
> > >
> >
> > Seriously? :-(
> >
>
> Please see SOLR-14022.
>
>
> > So next steps will be kicking out Cloud and go back to single node or
> what?
> >
>
> Not something we've considered yet.
>
>
> > Why don't you just freeze the whole Solr development and switch to
> Elastic?
> >
>
> Not something we've considered yet.
>
>
> >
> > >
> > >
> > > All of these are scheduled to be removed in a future 9.x release.
> > >
> > > It was decided that these components did not meet the standards of
> > quality
> > > and support that we wish to ensure for all components we ship. Some of
> > > these also relied on design patterns that we no longer recommend for
> use
> > in
> > > critical production environments.
> > >
> > > If you rely on these features, you are encouraged to try out community
> > > supported versions of these, where available [0]. Where such community
> > > support is not available, we encourage you to participate in the
> > migration
> > > of these components into community supported packages and help continue
> > the
> > > development. We envision that using packages for these components via
> > > package manager will actually make it easier for users to use such
> > features.
> > >
> > > Regards,
> > >
> > > Ishan Chattopadhyaya
> > >
> > > (On behalf of the Apache Lucene/Solr PMC)
> > >
> > > [0] -
> > >
> >
> https://cwiki.apache.org/confluence/display/SOLR/Community+supported+packages+for+Solr
> > >
> > > On Wed, Jul 15, 2020 at 2:30 PM Bruno Roustant <
> bruno.roust...@gmail.com
> > >
> > > wrote:
> > >
> > >> The Lucene PMC is pleased to announce the release of Apache Solr
> 8.6.0.
> > >>
> > >>
> > >> Solr is the popular, blazing fast, open source NoSQL search platform
> > from
> > >> the Apache Lucene project. Its major features include powerful
> full-text
> > >> search, hit highlighting, faceted search, dynamic clustering, database
> > >> integration, rich document handling, and geospatial search. Solr is
> > highly
> > >> scalable, providing fault tolerant distributed search and indexing,
> and
> > >> powers the search and navigation features of many of the world's
> largest
> > >> internet sites.
> > >>
> > >>
> > >> Solr 8.6.0 is available for immediate download at:
> > >>
> > >>
> > >>   
> > >>
> > >>
> > >> ### Solr 8.6.0 Release Highlights:
> > >>
> > >>
> > >>  * Cross-Collection Join Queries: Join queries can now work
> > >> cross-collection, even when shared or when spanning nodes.
> > >>
> > >>  * Search: Performance improvement for some types of queries when
> exact
> > >> hit count isn't needed by using BlockMax WAND algorithm.
> > >>
> > >>  * Streaming Expression: Percentiles and standard deviation
> aggregations
> > >> added to stats, facet and time series.  Streaming expressions added to
> > >> /export handler.  Drill Streaming Expression for efficient and
> accurate
> > >> high cardinality aggregation.
> > >>
> > >>  * Package manager: Support for cluster (CoreContainer) level plugins.
> > >>
> > >>  * Health Check: HealthCheckHandler can now require that all cores are
> > >> healthy before returning OK.
> > >>
> > >>  * Zookeeper read API: A read API at /api/cluster/zk/* to fetch raw ZK
> > >> data and view contents of a ZK directory.
> > >>
> > >>  * Admin UI: New panel with security info in admin UI's dashboard.
> > >>
> > >>  * Query DSL: Support for {param:ref} and {bool: {excludeTags:""}}
> > >>
> > >>  * Ref Guide: Major redesign of Solr's documentation.
> > >>
> > >>
> > >> Please read CHANGES.txt for a full list of new features and changes:
> > >>
> > >>
> > >>   

Re: +(-...) vs +(*:* -...) vs -(+...)

[Houston Putman]

Jochen,

For the standard query parser, pure negative queries (no positive query in
front of it, such as "*:*") are only allowed as a top level clause, so not
nested within parenthesis.

Check the second bullet point of the this section of the Ref Guide page for
the Standard Query Parser.


For the edismax query parser, pure negative queries are allowed to be
nested within parenthesis. Docs can be found in the Ref Guide page for the
eDismax Query Parser.


- Houston

On Thu, May 21, 2020 at 2:25 PM Jochen Barth 
wrote:

> Dear reader,
>
> why does +(-x_ss:y) finds 0 docs,
>
> while -(+x_ss:y) finds many docs?
>
> Ok... +(*:* -x_ss:y) works, too, but I'm a bit surprised.
>
> Kind regards, J. Barth
>
>

Re: Problems when Upgrading from Solr 7.7.1 to 8.5.0

[Houston Putman]

Hello Ludger,

I don't have answers to all of your questions, but for #2 (Incorrect Load
Balancing) it is a bug that will be fixed in 8.6. You can find more info at
SOLR-14471 .

- Houston

On Mon, May 11, 2020 at 8:16 AM Ludger Steens 
wrote:

> Hi all,
>
> we recently upgraded our SolrCloud cluster from version 7.7.1 to version
> 8.5.0 and ran into multiple problems.
> In the end we had to revert the upgrade and went back to Solr 7.7.1.
>
> In our company we are using Solr since Version 4 and so far, upgrading
> Solr to a newer version was possible without any problems.
> We are curious if others are experiencing the same kind of problems and if
> these are some known issues. Or maybe we did something wrong and missed
> something when upgrading?
>
>
> 1. Network issues when indexing documents
> ===
>
> Our collection contains roughly 150 million documents.  When we re-created
> the collection and re-indexed all documents, we regularly experienced
> network problems that causes our loader application to fail.
> The Solr log always contains an IOException Exception:
>
> ERROR
> (updateExecutor-5-thread-1338-processing-x:PSMG_CI_2020_04_15_10_07_04_sha
> rd6_replica_n22 r:core_node25 null n:solr2:8983_solr
> c:PSMG_CI_2020_04_15_10_07_04 s:shard6) [c:PSMG_CI_2020_04_15_10_07_04
> s:shard6 r:core_node25 x:PSMG_CI_2020_04_15_10_07_04_shard6_replica_n22]
> o.a.s.u.ErrorReportingConcurrentUpdateSolrClient Error when calling
> SolrCmdDistributor$Req: cmd=add{,id=(null)}; node=StdNode:
> http://solr1:8983/solr/PSMG_CI_2020_04_15_10_07_04_shard6_replica_n20/ to
> http://solr1:8983/solr/PSMG_CI_2020_04_15_10_07_04_shard6_replica_n20/ =>
> java.io.IOException: java.io.IOException: cancel_stream_error
>  at
> org.eclipse.jetty.client.util.DeferredContentProvider.flush(DeferredConten
> tProvider.java:197)
>  java.io.IOException: java.io.IOException: cancel_stream_error
>  at
> org.eclipse.jetty.client.util.DeferredContentProvider.flush(DeferredConten
> tProvider.java:197) ~[jetty-client-9.4.24.v20191120.jar:9.4.24.v20191120]
>  at
> org.eclipse.jetty.client.util.OutputStreamContentProvider$DeferredOutputSt
> ream.flush(OutputStreamContentProvider.java:151)
> ~[jetty-client-9.4.24.v20191120.jar:9.4.24.v20191120]
>  at
> org.eclipse.jetty.client.util.OutputStreamContentProvider$DeferredOutputSt
> ream.write(OutputStreamContentProvider.java:145)
> ~[jetty-client-9.4.24.v20191120.jar:9.4.24.v20191120]
>  at
> org.apache.solr.common.util.FastOutputStream.flush(FastOutputStream.java:2
> 16) ~[solr-solrj-8.5.0.jar:8.5.0 7ac489bf7b97b61749b19fa2ee0dc46e74b8dc42
> - romseygeek - 2020-03-1309:38:26]
>  at
> org.apache.solr.common.util.FastOutputStream.flushBuffer(FastOutputStream.
> java:209) ~[solr-solrj-8.5.0.jar:8.5.0
> 7ac489bf7b97b61749b19fa2ee0dc46e74b8dc42 - romseygeek - 202003-13
> 09:38:26]
>  at
> org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:172)
> ~[solr-solrj-8.5.0.jar:8.5.0 7ac489bf7b97b61749b19fa2ee0dc46e74b8dc42 -
> romseygeek - 2020-03-13 09:3826]
>
> After the Exception the collection usually was in a degraded state for
> some time and shards try to recover and sync with the leader.
>
> In the Solr changelog we saw that one major change from 7.x to 8.x was
> that Solr now uses HTTP/2 instead of HTTP/1.1. So we tried to disable
> HTTP/2 by setting the system property solr.http1=true.
> That did make the indexing process a LOT more stable but we still saw a
> IOExceptions from time to time. Disabling HTTP/2 did not completely fix
> the problem.
>
> ERROR
> (updateExecutor-5-thread-9310-processing-x:PSMG_BOM_2020_04_28_05_00_11_sh
> ard7_replica_n24 r:core_node27 null n:solr3:8983_solr
> c:PSMG_BOM_2020_04_28_05_00_11 s:shard7) [c:PSMG_BOM_2020_04_28_05_00_11
> s:shard7 r:core_node27 x:PSMG_BOM_2020_04_28_05_00_11_shard7_replica_n24]
> o.a.s.u.ErrorReportingConcurrentUpdateSolrClient Error when calling
> SolrCmdDistributor$Req: cmd=add{,id=5141653a-e33a-4b60-856d-7aa2ce73dee7};
> node=ForwardNode:
> http://solr2:8983/solr/PSMG_BOM_2020_04_28_05_00_11_shard6_replica_n22/ to
> http://solr2:8983/solr/PSMG_BOM_2020_04_28_05_00_11_shard6_replica_n22/ =>
> java.io.IOException: java.io.EOFException:
> HttpConnectionOverHTTP@9dc7ad1::SocketChannelEndPoint@2d20213b{solr2/10.0.
> 0.216:8983<->/10.0.0.193:38728,ISHUT,fill=-,flush=-,to=5/60}{io=0/0,ki
> o=0,kro=1}->HttpConnectionOverHTTP@9dc7ad1(l:/10.0.0.193:38728 <->
> r:solr2/10.0.0.216:8983,closed=false)=>HttpChannelOverHTTP@47a242c3(exchan
> ge=HttpExchange@6ffd260f req=PENDING/null@null
> res=PENDING/null@null)[send=HttpSenderOverHTTP@17e056f9(req=CONTENT,snd=ID
> LE,failure=null)[HttpGenerator@3b6594c7{s=COMMITTED}],recv=HttpReceiverOve
> rHTTP@6e847d32(rsp=IDLE,failure=null)[HttpParser{s=CLOSED,0 of -1}]]
> at
> org.eclipse.jetty.client.util.DeferredContentProvider.flush(DeferredConten
> 

Re: Slow Query in Solr 8.3.0

[Houston Putman]

Hey Vishal,

That's quite a large query. But I think the problem might be completely
unrelated. Are any of the return fields multi-valued? There was a major bug
(SOLR-14013 <https://issues.apache.org/jira/browse/SOLR-14013>) in
returning multi-valued fields that caused trivial queries to take around 30
seconds or more. You should be able to fix this by upgrading to 8.4, which
has the fix included, if you are in fact using multivalued fields.

- Houston Putman

On Wed, May 13, 2020 at 7:02 AM vishal patel 
wrote:

> I am upgrading Solr 6.1.0 to Solr 8.3.0.
>
> I have created 2 shards and one form collection in Solr 8.3.0. My schema
> file of form collection is same as Solr 6.1.0. Also Solr config file is
> same.
>
> I am executing below URL
> http://193.268.300.145:8983/solr/forms/select?q=(+(doctype:Apps AND
> ((allowed_roles:(2229130)) AND ((is_draft:true AND ((distribution_list:24
> OR draft_form_all_org_allowed_roles:(2229130)) OR
> (draft_form_own_org_allowed_roles:(2229130) AND
> msg_distribution_org_list:13))) OR (is_draft:false AND is_public:true AND
> (is_controller_based:false OR msg_type_id:(1 3))) OR ((allowed_users:24) OR
> (is_draft:false AND (is_public:false OR is_controller_based:true) AND
> ((distribution_list:24 OR private_form_all_org_allowed_roles:(2229130)) OR
> (private_form_own_org_allowed_roles:(2229130) AND
> msg_distribution_org_list:13)) AND appType:2 AND
> is_formtype_active:true -status_id:(23) AND (is_draft:false OR
> msg_type_id:1) AND instance_group_id:(2289710) AND project_id:(2079453) AND
> locationId:(9696 9694))) AND +msg_id:(10519539^3835 10519540^3834
> 10523575^3833 10523576^3832 10523578^3831 10525740^3830 10527812^3829
> 10528779^3828 10528780^3827 10530141^3826 10530142^3825 10530143^3824
> 10530147^3823 10525725^3822 10525716^3821 10526659^3820 10526661^3819
> 10529460^3818 10529461^3817 10530338^3816 10531331^3815 10521069^3814
> 10514233^3813 10514235^3812 10514236^3811 10514818^3810 10518287^3809
> 10518289^3808 10518292^3807 10518291^3806 10514823^3805 3117146^3804
> 3120673^3803 10116612^3802 10117480^3801 10117641^3800 10117810^3799
> 10119703^3798 10128983^3797 10229892^3796 10232225^3795 10233021^3794
> 10237712^3793 10237744^3792 10239494^3791 10239499^3790 10239500^3789
> 10243233^3788 10243234^3787 10305946^3786 10305977^3785 10305982^3784
> 10306994^3783 10306997^3782 10306999^3781 10308101^3780 10308772^3779
> 10308804^3778 10309685^3777 10309820^3776 10309821^3775 10310633^3774
> 10310634^3773 10311207^3772 10311210^3771 10352946^3770 10352947^3769
> 10353164^3768 10353171^3767 10353176^3766 10353956^3765 10354791^3764
> 10354792^3763 10354794^3762 10354798^3761 10355333^3760 10355353^3759
> 10355406^3758 10355995^3757 10356008^3756 10358933^3755 10358935^3754
> 10359420^3753 10359426^3752 10421223^3751 10421224^3750 10421934^3749
> 10422864^3748 10422865^3747 10426444^3746 10426446^3745 10428470^3744
> 10430357^3743 10430366^3742 10431990^3741 10490422^3740 10490430^3739
> 10490742^3738 10490745^3737 10491552^3736 10492344^3735 10492964^3734
> 10493965^3733 10494657^3732 10494660^3731 3121708^3730 3122606^3729
> 3124424^3728 3125051^3727 3125782^3726 3125793^3725 3127499^3724
> 3127600^3723 3127615^3722 3129535^3721 3131364^3720 3131377^3719
> 3132062^3718 3133668^3717 3134414^3716 10131445^3715 10133209^3714
> 10135640^3713 10136424^3712 10137129^3711 10137168^3710 10244270^3709
> 10244324^3708 10244326^3707 10248136^3706 10248137^3705 10248138^3704
> 10258595^3703 10259267^3702 10259966^3701 10259967^3700 10260700^3699
> 10260701^3698 10262790^3697 10264386^3696 10264536^3695 10264961^3694
> 10265098^3693 10265099^3692 10311754^3691 10312638^3690 10312639^3689
> 10312640^3688 10313909^3687 10313910^3686 10314024^3685 10314659^3684
> 10314691^3683 10314696^3682 10315395^3681 10315426^3680 10359451^3679
> 10359835^3678 10361077^3677 10361085^3676 10361277^3675 10361289^3674
> 10361824^3673 10362431^3672 10362434^3671 10363618^3670 10365316^3669
> 10365322^3668 10365327^3667 10433969^3666 10435946^3665 10435963^3664
> 10437695^3663 10437697^3662 10437698^3661 10437703^3660 10438761^3659
> 10438763^3658 10439721^3657 10439728^3656 10496118^3655 10496281^3654
> 10496289^3653 10496294^3652 10496296^3651 10496570^3650 10496582^3649
> 10496626^3648 10497518^3647 10497522^3646 10497530^3645 10498717^3644
> 10498722^3643 10499254^3642 10499256^3641 10500374^3640 10500382^3639
> 10507062^3638 10507061^3637 3134424^3636 3135192^3635 3135284^3634
> 3135293^3633 3139529^3632 3140767^3631 3141525^3630 3141681^3629
> 3141690^3628 3142537^3627 3143664^3626 3144581^3625 3145417^3624
> 3145862^3623 3146382^3622 3147405^3621 10299450^3620 10299963^3619
> 10341581^3618 10343393^3617 10344195^3616 10345911^3615 10345920^3614
> 10345922^3613 10391985^3612 10391986^

Re: Checking my understanding of SOLR_HOME

[Houston Putman]

1. You are correct about the SOLR_HOST parameter.

2. Your SOLR_PORT variable should be 8983, as that's what solr will be
listening on. However you want to make sure to use port 8985 in the
solr.xml so that the node advertises itself as using that port in the
liveNodes.


  
${hostPort:8985}
  

With this setup you can pass in the external port as -DhostPort=<>


On Fri, Mar 27, 2020 at 1:17 PM Eric Pugh 
wrote:

> I am struggling with using the zkHost and the JDBC end point (
> https://lucene.apache.org/solr/guide/6_6/parallel-sql-interface.html#jdbc-driver)
> and I believe it’s because when I deploy, it gets a IP address that is
> internal to the network accessible, but accessible externally via DNS name:
>
> http://quepid-solr.dev.o19s.com:8985/solr/#/~cloud?view=tree
>
> I’m also using Docker, so the internal :8983 gets mapped to the external
> :8985 port.
>
> I *think* what I need to do is:
>
> 1) Use the SOLR_HOST parameter to make sure the hostname is “
> quepid-solr.dev.o19s.com” in my startup Script.
> 2) Set the environment variable SOLR_PORT to be 8985 instead of using the
> Docker mapping of ports.
>
> If this is correct understanding, then I think adding a bit more
> documentation to
> https://lucene.apache.org/solr/guide/8_4/taking-solr-to-production.html#solr-hostname
> would be useful, and happy to add a documentation PR as it’s not super
> clear to me.
>
> Eric
> ___
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com <
> http://www.opensourceconnections.com/> | My Free/Busy <
> http://tinyurl.com/eric-cal>
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed <
> https://www.packtpub.com/big-data-and-business-intelligence/apache-solr-enterprise-search-server-third-edition-raw>
>
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
>
>

Re: Backups with only 1 machine having access to remote storage?

[Houston Putman]

>From my experience, you need all nodes to have access to the shared storage.
Solr will pick which nodes should write each shard's data, and you do not
have a lot of control over which nodes are selected.
This is why in the documentation it says that the backup must be written to
NFS or HDFS.
Solr won't try to retrieve the other replicas over the network.

I think you will actually get an error back when not every node is able to
see the path where the backup should be written.
But even if you don't receive an error, the backup will not work.

- Houston



On Thu, Feb 20, 2020 at 11:26 AM Koen De Groote 
wrote:

> Hello all,
>
> I've recently set up backups, using solr 7.6
>
> My setup has 3 replicas per collection and several collections. Not all
> collections or replicas are present on all hosts.
>
> That being said, I run the backup command from 1 particular host and only
> that host has access to the mount on which the backup data will be written.
>
> This means that the host writing the backup data doesn't have all the data
> on its local filesystem.
>
> Is this a problem?
>
> By which I mean: will data not present on that host be retrieved over the
> network?
>
> What happens in this case?
>
> Kind regards,
> Koen De Groote
>

Re: problem using Http2SolrClient with solr 8.3.0

[Houston Putman]

Are you overriding the Jetty version in your application using SolrJ?

On Wed, Nov 27, 2019 at 4:00 PM Odysci  wrote:

> Hi,
> I have a solr cloud setup using solr 8.3 and SolrJj, which works fine using
> the HttpSolrClient as well as the CloudSolrClient. I use 2 solr nodes with
> 3 Zookeeper nodes.
> Recently I configured my machines to handle ssl, http/2 and then I tried
> using in my java code the Http2SolrClient supported by SolrJ 8.3.0, but I
> got the following error at run time upon instantiating the Http2SolrClient
> object:
>
> Has anyone seen this problem?
> Thanks
> Reinaldo
> ===
>
> Oops: NoClassDefFoundError
> Unexpected error : Unexpected Error, caused by exception
> NoClassDefFoundError: org/eclipse/jetty/client/api/Request
>
> play.exceptions.UnexpectedException: Unexpected Error
> at play.jobs.Job.onException(Job.java:180)
> at play.jobs.Job.call(Job.java:250)
> at Invocation.Job(Play!)
> Caused by: java.lang.NoClassDefFoundError:
> org/eclipse/jetty/client/api/Request
> at
>
> org.apache.solr.client.solrj.impl.Http2SolrClient$AsyncTracker.(Http2SolrClient.java:789)
> at
>
> org.apache.solr.client.solrj.impl.Http2SolrClient.(Http2SolrClient.java:131)
> at
>
> org.apache.solr.client.solrj.impl.Http2SolrClient$Builder.build(Http2SolrClient.java:833)
> ... more
> Caused by: java.lang.ClassNotFoundException:
> org.eclipse.jetty.client.api.Request
> at
>
> java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
> at
>
> java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
> at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
> ... 16 more
> ==
>

Re: Changing the IP or a SOLR or Zookeeper host

[Houston Putman]

If you are changing the IP of a host running Zookeeper, it can be an issue.
It depends on the version of zookeeper that you are using. There was an
issue with ZK not re-resolving IP addresses on connection errors, but it
was fixed in 3.4.13 (
https://issues.apache.org/jira/plugins/servlet/mobile#issue/ZOOKEEPER-2184
). If you are using 3.5.x, you should be safe as the fix was made there as
well.

You should be fine doing this for Solr nodes, but Java has weird default
settings for caching DNS, where it can cache a DNS resolution infinitely
and never refresh it. Therefore I'd say you're safest if you do a rolling
restart of your cluster, and any clients using solrJ. (Unless you make sure
that you have the correct Java DNS cache settings)

On Tue, Nov 19, 2019, 2:52 PM dshih  wrote:

> If a SOLR or Zookeeper host is replaced such that the hostname comes back
> but
> it now resolves to a different IP, are SOLR nodes and SOLRJ clients
> expected
> to just continue working?
>
> I know that in this case replicas need to be re-created (and stale ones
> deleted).  But I'm wondering if SOLRJ clients and/or other SOLR nodes need
> to be restarted to pick up the hostname->IP change.
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>

Re: Solr healthcheck fails all the time

[Houston Putman]

Hello,

Could you provide some more information about your cloud, for example:

   - The number of requests that it handles per minute
   - How much data you are indexing
   - If there is any memory pressure

The ping handler merely sends a query to the collection and makes sure that
it responds healthily. Can you check your schema to see what the query is
that it is sending? The ping query may be more expensive than you think.
Also is the ping using distrib=true or false in the query?

On Wed, Nov 6, 2019 at 6:45 PM amruth  wrote:

> I am running Solr Cloud 6.6 and all the nodes fail healthcheck too
> frequently
> with *Read Timed out * error. Here is the stacktrace,
>
> http://solr-host1:8983/solr/collection1/admin/ping is DOWN, error:
> HTTPConnectionPool(host='solr-host1', port=8983): Read timed out. (read
> timeout=1). Connection failed after 1001 ms
>
> Can someone please say why it fails all the time?(at least once every
> 10min)
>
>
>
> --
> Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>