Issue in Solr upgrade from 5.x to 8.x
Hi All, We are presently using solr 5.3.0 and planning to migrate to version 8.4.1 .I tried using index upgrader for the same . I am able to upgrade 5.3.0->6.6.6 and 6.6.6 to 7.7.3.But when I try upgrading 7.7.3 index to 8.4.1,I keep getting following error : This index was initially created with Lucene 6.x while the current version is 8.4.1 and Lucene only supports reading the current and previous major versions.. This version of Lucene only supports indexes created with release 7.0 and later. Is there any workaround to do this ,without re-indexing?Any help/suggestions would be much appreciated. Thanks Anchal
Solr not starting after enabling SSL
Hi All,
We recently migrated our existing solr(version 5.3.0) from AIX OS server to
Linux based server.And it works fine(http solr) .
RHEL version 7.6
Java version 1.8(IBM Java)
But now ,when trying to enable SSL over same ,the solr doesnt start after
enabling SSL.
It says "Address already in use" despite there being no solr up .
2019-04-03 06:29:29.892 WARN (main) [ ] o.e.j.u.c.AbstractLifeCycle FAILED
ServerConnector@cdf341f{SSL-http/1.1}{0.0.0.0:8983}: java.net.BindException:
Address already in use
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:460)
at sun.nio.ch.Net.bind(Net.java:452)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:253)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:86)
at
org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1255)
at
java.security.AccessController.doPrivileged(AccessController.java:647)
at
org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:508)
at org.eclipse.jetty.start.Main.invokeMain(Main.java:321)
at org.eclipse.jetty.start.Main.start(Main.java:817)
at org.eclipse.jetty.start.Main.main(Main.java:112)
Steps used to enable solr SSL
:https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html (Same was used over
AIX server's solr to enable SSL and we were successful there)
Any suggestion would be highly appreciated!!
Thanks & Regards,
-
Anchal Sharma
solr SSL encryption degardes solr performance
Hi All, We had recently enabled SSL on solr. But afterwards ,our application performance has degraded significantly i.e the time for the source application to fetch a record from solr has increased from approx 4 ms to 200 ms(this is for a single record) .This amounts to a lot of time ,when multiple calls are made to solr. Has any one experienced this ,and please share if some one has any suggestion . Thanks & Regards, - Anchal Sharma
Re: solr is using TLS1.0
Hi Hendrick This did the trick .Overriding default TLS version for IBM Java enabled TLS 1.2 for solr . Thank you Hendrick /Shawn for your help and suggestions. Thanks & Regards, - Anchal Sharma From: Hendrik Haddorp To: solr-user@lucene.apache.org Date: 22-11-2018 12:53 Subject:Re: solr is using TLS1.0 Hi Anchal, the IBM JVM behaves differently in the TLS setup then the Oracle JVM. If you search for IBM Java TLS 1.2 you find tons of reports of problems with that. In most cases you can get around that using the system property "com.ibm.jsse2.overrideDefaultTLS" as documented here: https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.security.component.80.doc/security-component/jsse2Docs/matchsslcontext_tls.html regards, Hendrik On 22.11.2018 07:25, Anchal Sharma2 wrote: > > Hi Shawn , > > Thanks for your reply . > > Here are the details abut java we are using : > java version "1.8.0_151" > IBM J9 VM (build 2.9, JRE 1.8.0 AIX ppc64-64 Compressed References > 20171102_369060 (JIT enabled, AOT enabled) > I have already patched the policy jars . > > And I tried to comment out the ciphers ,protocol entries in > jetty-ssl.xml ,but it did not work for me .I also tried to use an > "IncludeCipherSuites" entry to include a cipher I wanted to include > ,but it did not work either .I started getting > SSL_ERROR_INTERNAL_ERROR_ALERT and ssl_error_no_cypher_overlap errors > on my console URL.I tried this in solr 7.3.1 version ,so jetty version > must also be relatively new. > > Do you think java might not be letting me enable TLS1.2? > > Thanks & Regards, > - > Anchal Sharma > > > Inactive hide details for Shawn Heisey ---21-11-2018 05:28:50---On > 11/20/2018 3:02 AM, Anchal Sharma2 wrote: > I have enabled Shawn > Heisey ---21-11-2018 05:28:50---On 11/20/2018 3:02 AM, Anchal Sharma2 > wrote: > I have enabled SSL for solr using steps mentioned o > > From: Shawn Heisey > To: solr-user@lucene.apache.org > Date: 21-11-2018 05:28 > Subject: Re: solr is using TLS1.0 > > > > > > On 11/20/2018 3:02 AM, Anchal Sharma2 wrote: > > I have enabled SSL for solr using steps mentioned over Lucene > > website .And though solr console URL is now secure(https) ,it is still > > using TLS v1.0. > > I have tried few things to force SSL to use TLS1.2 protocol ,but > they > > have not worked for me . > > > > While trying to do same ,I have observed solr itself does not offer any > > solr property to specify cipher ,algorithm or TLS version . > > > > Following things have been tried : > > 1.key store /trust store for solr to enable SSL with different key > > algorithm ,etc combinations for the certificates > > 2.different solr versions for step 1(solr 5.x,6.x,7.x-we are using solr > > 5.3 currently) > > 3.using java version 1.8 and adding solr certificate in java keystore to > > enforce TLS1.2 > > Solr lets Java and Jetty handle TLS. Solr itself doesn't get involved > except to provide information to other software. > > There are a whole lot of versions of Java 8, and at least three vendors > for it. The big names are Oracle, IBM, and OpenJDK. What vendor and > exact version of Java are you running? What OS is it on? Do you have > the "unlimited JCE" addition installed in your Java and enabled? If > your Java version is new enough, you won't need to mess with JCE. See > this page: > > https://golb.hplar.ch/2017/10/JCE-policy-changes-in-Java-SE-8u151-and-8u152.html > > Solr 5.3 ships with Jetty 9.2.11, which is considered very outdated by > the Jetty project -- released well over three years ago. From the > perspective of the Solr project, version 5.3 is also very old -- two > major versions behind what's current, and also released three years ago. > > Jetty 9.2 is up to 9.2.26. The current version is Jetty 9.4.14. The > latest version of Solr (7.5.0) is shipping with Jetty 9.4.11. I think > Jetty will likely be upgraded to the latest release for Solr 7.6.0. > > Have you made any changes to the Jetty config, particularly > jetty-ssl.xml? One thing you might try, although I'll warn you that it > may make no difference at all, is to remove the parts of that config > file that exclude certain protocols and ciphers, letting Jetty decide > for itself what it should use. Recent versions of Jetty and Java have > very good defaults. I do not know whether Jetty 9.2.11 (included with > Solr 5.3, as mentioned) has good defaults or not. > > Thanks, > Shawn > > > > >
Re: solr is using TLS1.0
Hi Shawn , Thanks for your reply . Here are the details abut java we are using : java version "1.8.0_151" IBM J9 VM (build 2.9, JRE 1.8.0 AIX ppc64-64 Compressed References 20171102_369060 (JIT enabled, AOT enabled) I have already patched the policy jars . And I tried to comment out the ciphers ,protocol entries in jetty-ssl.xml ,but it did not work for me .I also tried to use an "IncludeCipherSuites" entry to include a cipher I wanted to include ,but it did not work either .I started getting SSL_ERROR_INTERNAL_ERROR_ALERT and ssl_error_no_cypher_overlap errors on my console URL.I tried this in solr 7.3.1 version ,so jetty version must also be relatively new. Do you think java might not be letting me enable TLS1.2? Thanks & Regards, - Anchal Sharma From: Shawn Heisey To: solr-user@lucene.apache.org Date: 21-11-2018 05:28 Subject:Re: solr is using TLS1.0 On 11/20/2018 3:02 AM, Anchal Sharma2 wrote: > I have enabled SSL for solr using steps mentioned over Lucene > website .And though solr console URL is now secure(https) ,it is still > using TLS v1.0. > I have tried few things to force SSL to use TLS1.2 protocol ,but they > have not worked for me . > > While trying to do same ,I have observed solr itself does not offer any > solr property to specify cipher ,algorithm or TLS version . > > Following things have been tried : > 1.key store /trust store for solr to enable SSL with different key > algorithm ,etc combinations for the certificates > 2.different solr versions for step 1(solr 5.x,6.x,7.x-we are using solr > 5.3 currently) > 3.using java version 1.8 and adding solr certificate in java keystore to > enforce TLS1.2 Solr lets Java and Jetty handle TLS. Solr itself doesn't get involved except to provide information to other software. There are a whole lot of versions of Java 8, and at least three vendors for it. The big names are Oracle, IBM, and OpenJDK. What vendor and exact version of Java are you running? What OS is it on? Do you have the "unlimited JCE" addition installed in your Java and enabled? If your Java version is new enough, you won't need to mess with JCE. See this page: https://golb.hplar.ch/2017/10/JCE-policy-changes-in-Java-SE-8u151-and-8u152.html Solr 5.3 ships with Jetty 9.2.11, which is considered very outdated by the Jetty project -- released well over three years ago. From the perspective of the Solr project, version 5.3 is also very old -- two major versions behind what's current, and also released three years ago. Jetty 9.2 is up to 9.2.26. The current version is Jetty 9.4.14. The latest version of Solr (7.5.0) is shipping with Jetty 9.4.11. I think Jetty will likely be upgraded to the latest release for Solr 7.6.0. Have you made any changes to the Jetty config, particularly jetty-ssl.xml? One thing you might try, although I'll warn you that it may make no difference at all, is to remove the parts of that config file that exclude certain protocols and ciphers, letting Jetty decide for itself what it should use. Recent versions of Jetty and Java have very good defaults. I do not know whether Jetty 9.2.11 (included with Solr 5.3, as mentioned) has good defaults or not. Thanks, Shawn
solr is using TLS1.0
Hi All, I have enabled SSL for solr using steps mentioned over Lucene website .And though solr console URL is now secure(https) ,it is still using TLS v1.0. I have tried few things to force SSL to use TLS1.2 protocol ,but they have not worked for me . While trying to do same ,I have observed solr itself does not offer any solr property to specify cipher ,algorithm or TLS version . Following things have been tried : 1.key store /trust store for solr to enable SSL with different key algorithm ,etc combinations for the certificates 2.different solr versions for step 1(solr 5.x,6.x,7.x-we are using solr 5.3 currently) 3.using java version 1.8 and adding solr certificate in java keystore to enforce TLS1.2 4.various kind of keystores like JKS,PKCS12,etc Can anyone offer any suggestions on same ?I have not been able to find much about same niofficial site. Thanks & Regards, - Anchal Sharma
Re: Question regarding TLS version for solr
Hi Chris, Thanks a lot for sharing the steps . I tried few of them .Actually we already have been using solr in our application since an year or so .We just want to encrypt it to use secure solr now .So ,I followed the steps where you have created the certificates ,etc .But when I go to start the solr back ,it doesnt start . We are using zookeeper .Following is the error I get ,on running solr start command. Command:./solr -c -m 1g -p 8984 -z :2181 -s Error: lsof 4.55 (latest revision at ftp://vic.cc.purdue.edu/pub/tools/unix/lsof) usage: [-?abhlnNoOPRstUvVX] [-c c] [+|-d s] [+|-D D] [+|-f[cfgGn]] [-F [f]] [-g [s]] [-i [i]] [+|-L [l]] [-m m] [+|-M] [-o [o]] [-p s] [+|-r [t]] [-S [t]] [-T [t]] [-u s] [+|-w] [--] [names] Use the ``-h'' option to get more help information. Still not seeing Solr listening on 8984 after 30 seconds! at java.security.KeyStore.load(KeyStore.java:1456) at org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:55) at org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:871) at org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:273) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:64) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:256) at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.server.Server.doStart(Server.java:366) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1255) at java.security.AccessController.doPrivileged(AccessController.java:594) at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1174) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:90) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.eclipse.jetty.start.Main.invokeMain(Main.java:321) at org.eclipse.jetty.start.Main.start(Main.java:817) at org.eclipse.jetty.start.Main.main(Main.java:112) 2018-05-24 09:05:16.714 INFO (zkCallback-3-thread-1-processing-n:9.109.122.113:8984_solr) [ ] o.a.s.c.c.ZkStateReader A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1) 2018-05-24 09:05:17.018 INFO (zkCallback-3-thread-1-processing-n:9.109.122.113:8984_solr) [ ] o.a.s.c.c.ZkStateReader Updated cluster state version to 9702 2018-05-24 09:05:17.153 INFO (coreLoadExecutor-7-thread-2-processing-n:9.109.122.113:8984_solr) [c:document r:core_node1 x:document] o.a.s.u.SolrIndexConfig IndexWriter infoStream solr logging is enabled [\] sleep: bad character in argument Thanks & Regards, - Anchal Sharma e-Pricer Development ES Team Mobile: +9871290248 -Christopher Schultz <ch...@christopherschultz.net> wrote: - To: solr-user@lucene.apache.org From: Christopher Schultz <ch...@christopherschultz.net> Date: 05/23/2018 07:29PM Subject: Re: Question regarding TLS version for solr -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Anchal, On 5/23/18 2:38 AM, Anchal Sharma2 wrote: > Thank you for replying .But ,I checked the java version solr using > ,and it is already version 1.8. > > @Christopher ,can you let me know what steps you followed for TLS > authentication on solr version 7.3.0. Sure. Here are my deployment notes. You may have to adjust them slightly for your environment. Note that we are using standalone Solr without any Zookeeper, clustering, etc. This is just about configuring a single instance. Also, this guide says 7.3.0, but 7.3.1 would be better as it contains a fix for a CVE. === CUT === ===
Re: Question regarding TLS version for solr
Hi Christopher /Shawn , Thank you for replying .But ,I checked the java version solr using ,and it is already version 1.8. @Christopher ,can you let me know what steps you followed for TLS authentication on solr version 7.3.0. Thanks & Regards, - Anchal Sharma e-Pricer Development ES Team Mobile: +9871290248 -Christopher Schultz <ch...@christopherschultz.net> wrote: - To: solr-user@lucene.apache.org From: Christopher Schultz <ch...@christopherschultz.net> Date: 05/17/2018 06:29PM Subject: Re: Question regarding TLS version for solr -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Shawn, On 5/17/18 4:23 AM, Shawn Heisey wrote: > On 5/17/2018 1:53 AM, Anchal Sharma2 wrote: >> We are using solr version 5.3.0 and have been trying to enable >> security on our solr .We followed steps mentioned on site >> -https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html .But >> by default it picks ,TLS version 1.0,which is causing an issue >> as our application uses TLSv 1.2.We tried using online resources >> ,but could not find anything regarding TLS enablement for solr . >> >> It will be a huge help if anyone can provide some suggestions as >> to how we can enable TLS v 1.2 for solr. > > The choice of ciphers and encryption protocols is mostly made by > Java. The servlet container might influence it as well. The only > servlet container that is supported since Solr 5.0 is the Jetty > that is bundled in the Solr download. > > TLS 1.2 was added in Java 7, and it became default in Java 8. If > you can install the latest version of Java 8 and make sure that it > has the policy files for unlimited crypto strength installed, > support for TLS 1.2 might happen automatically. There is no "default" TLS version for either the client or the server: the two endpoints always negotiate the highest mutual version they both support. The key agreement, authentication, and cipher suites are the items that are negotiated during the handshake. > Solr 5.3.0 is running a fairly old version of Jetty -- 9.2.11. > Information for 9.2.x versions is hard to find, so although I think > it probably CAN do TLS 1.2 if the Java version supports it, I can't > be absolutely sure. You'll need to upgrade Solr to get an upgraded > Jetty. I would be shocked if Jetty ships with its own crypto libraries; it should be using JSSE. Anchal, Java 1.7 or later is an absolute requirement if you want to use TLSv1.2 (and you SHOULD want to use it). I have recently spent a lot of time getting Solr 7.3.0 running with TLS mutual-authentication, but I haven't worked with the 5.3.x line. I can tell you have I've done things for my version, but they may need some adjustments for yours. - -chris -BEGIN PGP SIGNATURE- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlr9fKYACgkQHPApP6U8 pFh8lRAAmmvBMUSk35keW0OG0/SHpUy/ExJK69JGIKGwi96ddbz2yH8MG+OjjE3G GNq/o5+EMT7tP/nW6XuPQou5UQvA2nlA9jsskox3A+CqOH7e6cbSxfxIkTqf9YDl Kxr4J6mYjvTIjJAqLXGF+ghJfswS6RjZezDgo1PdSUox+gUOvmY61tlSjuYTaAYw vH1i1DRzb8PkkR4ULePF48Y4r5+ZYz/4ZwSvnJTTkyl97KCw93rZ/kI5v9p3cCHK Ycuwi/ZirO/VNf/9ruAOtgET3aojNfuNCX/A+vrSbJfiY7mXo05lYKN+eT80elQr X8OKQaqHP6haF2aNPHrqXGtY2YoiGrdyaGtrXkUHFDfXgQeOmlk/eSVWemcSsatk eEHSWW9NALMaalRAM7NuXQtgqq1badJhKysiJwSqFgcdgVKcSt8SsQ/09qTPjaNE Ce1/EHdR6j1hM0Bnv5Hzf85cZjM7PfLmh7P8fnUD5d8eSbBpeWYVBDsS+fXp8WWv FO5axbnSYIScOIz33i0UZyxpJgcsAkABLGghL6WWQSkfBf4ANgdTumS7K9Pn7Thz Uq+lD9QPEPWJ91Fc0gnCWtDAEIRjOyLLbYzgI4ebV5qo41GO1WDDHfQZEcqA0Vod +K8oAMD8nnwU+TprTFkjlQwbDnW1q1efTD6IrpEL5H7h6Xw2cgg= =RpO6 -END PGP SIGNATURE-
Question regarding TLS version for solr
Hi All, We are using solr version 5.3.0 and have been trying to enable security on our solr .We followed steps mentioned on site -https://lucene.apache.org/solr/guide/6_6/enabling-ssl.html .But by default it picks ,TLS version 1.0,which is causing an issue as our application uses TLSv 1.2.We tried using online resources ,but could not find anything regarding TLS enablement for solr . It will be a huge help if anyone can provide some suggestions as to how we can enable TLS v 1.2 for solr. Thanks & Regards, - Anchal Sharma
