Thanks a lot Erick. That was a big help. :)
-----Original Message----- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, June 28, 2016 5:47 PM To: solr-user Subject: Re: Beginer's questions 1> When running in SolrCloud mode the configs live on Zookeeper. When running locally the configs live on the local disk. The reason for this is that in Cloud mode, you might have replicas scattered all over the cluster on many machines (I've seen 1,000+ replicas in toto). You wouldn't want to have to manually keep all changes to configs in sync across that many nodes, so they're stored in ZK. See the upconfig/downconfig (well, zk -upconfig) in solr/bin for putting them up and down. Once a configset is up on Zookeeper, it can be used by one or more collections. 2> If you're content to have all your eggs in one basket there's no reason (and added complexity) to run in SolrCloud mode. However, if you want automatic fail-over for high availability or you need to scale to a higher QPS rate, then SolrCloud is usually much better. It's not _bad_ to run in SolrCloud mode with 1-shard collections, it's just a matter of whether maintaining Zookeeper is worth it. 3> Yes, all Solr installations handle queries and updates simultaneously. It doesn't matter whether it's a single shard/replica or many shards each with many replicas. And yeah, the nomenclature is confusing. node = where the Solr JVM is hosted. Often a physical machine is a node. The confusion here is that you can start several Solr JVMS on a single physical machine, each with a different port. I'll usually call each of these a node. instance = (usually) Solr running in a single JVM. As above there may be several instances on a single physical machine (node). NOTE: This is an especially fuzzy concept, this is my definition. collection = All the docs you are working with that you want to search with a single query. A collection may have one or more shards. shard = physical part of a larger logical index. Let's say you have two docs. doc1 could live on shard1 and doc2 on shard 2. core = call it an "administrative unit" in a single instance. This is where it gets really confusing since an instance can have 0-N cores. Each one is independent of all others at the lowest level for all they're in the same JVM. replica = a core that belongs to a collection. This tends to be used in SolrCloud only but that's just by convention. A single shard may have 1-N replicas scattered amongst many instances. Each and every one of them will have the same documents. In the example above, shard1_replica1, shard1_replica2.... will all have doc1. shard2_replica1, shard2_replica2 will have doc 2. Best, Erick On Tue, Jun 28, 2016 at 6:51 AM, Kostas <k...@dataverse.gr> wrote: > Hello Alexandre. > > Thanks for the help. > > 5) That Bug report seems very interesting. Thanks. :) I will try that > and let's hope the server requires the client to have a certificate, > unlike the current behavior I am getting when specifying the > certificate settings on Jetty. > > 6) For the client certificate, the tutorial > (https://cwiki.apache.org/confluence/display/solr/Enabling+SSL#Enablin > gSSL-Q > ueryusingcURL) says : >> To generate a self-signed certificate and a single key that will be >> used > to authenticate both the server and the client, we'll use the JDK > keytool command... > > It says to authenticate both, so there is not much room for an > alternative interpretation (I also have the impression that the -E > switch in cUrl is used for authentication on the server and not to > verify the servers certificate). > https://curl.haxx.se/docs/manpage.html >> -E, --cert (SSL) Tells curl to use the specified client certificate >> file > when getting a file with HTTPS, FTPS or another SSL-based protocol. > https://blogs.msdn.microsoft.com/kaushal/2012/02/17/client-certificate > s-vs-s > erver-certificates/ >> Client Certificates: Client certificates as the name indicates are >> used to > identify a client or a user. They are meant for authenticating the > client to the server. > > 7) Thanks. I think I understand a bit more now. :) > > Thanks a lot, > Kostas > > > > > > > -----Original Message----- > From: Alexandre Drouin [mailto:alexandre.dro...@orckestra.com] > Sent: Tuesday, June 28, 2016 4:09 PM > To: solr-user@lucene.apache.org > Subject: RE: Beginer's questions > > Hi, > > I'm also a Solr beginner but I think I can answer a few of your questions: > > 5 - There's a bug in solr.cmd related to SSL where the settings > defined in solr.in.cmd are ignored. You can see SOLR-8491 > (https://issues.apache.org/jira/browse/SOLR-8491) for more information > and fix. > > 6- From my understanding, the 'cUrl certificate' is only useful if > your Solr instance is using a self-signed certificate; it is used by > the client to validate that the server's certificate is valid. > > 7- The example (from your link) shows how to setups 2 Solr instances > on the same machine using different ports (-p) and data directories > (-s). When running in SolrCloud configuration, Solr's does not know > which protocol to use when communicating with other Solr instances and uses > HTTP by default. > You can change this behavior by changing the urlScheme cluster > property stored in ZooKeeper to HTTPS. > > > Alex > > > -----Original Message----- > From: Kostas [mailto:k...@dataverse.gr] > Sent: June 28, 2016 8:42 AM > To: solr-user@lucene.apache.org > Subject: RE: Beginer's questions > > Regarding the SSL question, it fails when I try this too : > > solr start -c -V ^ > -Dsolr.ssl.checkPeerName=false ^ > -Djavax.net.ssl.keyStorePassword=password ^ > -Djavax.net.ssl.trustStorePassword=password ^ > > -Djavax.net.ssl.keyStore="F:/Users/me/Downloads/solr-6.1.0/server/etc/ > solr-s > sl.keystore.jks" ^ > > -Djavax.net.ssl.trustStore="F:/Users/me/Downloads/solr-6.1.0/server/et > c/solr > -ssl.keystore.jks" > > But when I specify the exact same settings inside > `solr-6.1.0\server\etc\ jetty-ssl.xml` it works. > Do those two places do the same thing? > This is the failure I get when jetty-ssl.xml has a wrong password: > > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j > ava:62 > ) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess > orImpl > .java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.eclipse.jetty.start.Main.invokeMain(Main.java:214) > at org.eclipse.jetty.start.Main.start(Main.java:457) > at org.eclipse.jetty.start.Main.main(Main.java:75) > Caused by: java.io.IOException: Keystore was tampered with, or > password was incorrect > at > sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:780) > at > sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56) > at > sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.j > ava:22 > 4) > at > sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeySto > re.jav > a:70) > at java.security.KeyStore.load(KeyStore.java:1445) > at > org.eclipse.jetty.util.security.CertificateUtils.getKeyStore(Certifica > teUtil > s.java:52) > at > org.eclipse.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory. > java:1016) > at > org.eclipse.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java: > 332) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.j > ava:68) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle > .java:132) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCyc > le.java:114) > at > org.eclipse.jetty.server.SslConnectionFactory.doStart(SslConnectionFac > tory.j > ava:64) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.j > ava:68) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLif > eCycle > .java:132) > at > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerL > ifeCyc > le.java:114) > at > org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.j > ava:26 > 0) > at > org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetw > orkCon > nector.java:81) > at > org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.j > ava:68) > at org.eclipse.jetty.server.Server.doStart(Server.java:384) > at > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeC > ycle.j > ava:68) > at > org.eclipse.jetty.xml.XmlConfiguration$1.run(XmlConfiguration.java:1510) > at java.security.AccessController.doPrivileged(Native Method) > at > org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1435) > ... 7 more > Caused by: java.security.UnrecoverableKeyException: Password > verification failed > at > sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:778) > ... 30 more > > From a few tests that I've run, it seems that the `jetty-ssl.xml` > overrides the command-line arguments. > I haven't seen you mention anywhere that I should update > `jetty-ssl.xml` and the cmd-line doesn't seem to override those settings, so > what am I missing? > > Kostas > > > > > > > -----Original Message----- > From: Kostas [mailto:k...@dataverse.gr] > Sent: Tuesday, June 28, 2016 1:36 PM > To: solr-user@lucene.apache.org > Subject: Beginer's questions > > Hello. > > > > I have a ton of questions that I could use some answers. > > If someone can answer some of them it would be great. > > > > 1) I had problems making my Solr 6.1 setup use a fixed collection > schema. When I placed the schema.xml file in the filesystem as shown > here > <http://2.bp.blogspot.com/-zWmWPHKrmYA/URBOUoCYOLI/AAAAAAAAA8M/px33YlR > 5gCg/s 1600/croppedLargeFontSolrDirectoryStructure.jpg> my Solr > installation used to become corrupt (you cannot imagine how much time > I've spent on this thing). Later I came across this > <https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201509.mbo > x/%3CC > af2dzvvbh2bxrsyeoynrvnl2tjfydahijkakcbigazwpine...@mail.gmail.com%3E> > and this > <https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201509.mbo > x/%3CC > an4yxvf+1a6jsum3bsus2lo8sst3hbjuihr0-y6h4qjor57...@mail.gmail.com%3E> . > AN4YXvf+I > have done what it said (I have uploaded the complete configuration > onto ZooKeeper and then used it in a new collection) and it worked. > So, is this the new way of configuring a collection? I thought ZK was > only used for high scalability. If I run my Solr NOT in cloud mode, I > would still have to use the same method? > > 2) Is it bad to run Solr in cloud mode, even if you don't need high > scalability? Because I have configured it to run like that (cloud > mode) and I don't know if it will have any side-effects. I don't think > I need high scalability anyway. > > 3) When using only 1 core, 1 collection, 1 node, 1 shard (and I have > not yet cleared out all these terms by the way, even by looking at the > definitions for hours.) can Solr process incoming requests both for > querying and for posting new data in a multithreaded manner? I think > the answer here is yes, but it would be nice of you could confirm that to me. > > 4) I have tried to move the cores folder (SOLR_HOME) out of the default > location using the `solr.in.cmd` (I am on Windows) file by setting this: > set SOLR_HOME="F:\Users\me\Downloads\solr-6.1.0\solr-home" > I was expecting this to work out of the box, but turns out it doesn't. > The admin UI didn't seem to work quite well with that setting in > place. If I remember well, it failed when I tried to add a new core. > Am I missing something there? I generally start my Solr like that: > solr start -c -Dsolr.ssl.checkPeerName=false > -Djavax.net.ssl.keyStorePassword=password > -Djavax.net.ssl.trustStorePassword=password > > 5) I have tried to setup SSL on the Solr server. I have followed the > steps here > <https://cwiki.apache.org/confluence/display/solr/Enabling+SSL> , and > I have updated `solr.in.cmd` with all the settings. It didn't seem to > work (for some reason it didn't get the password correctly) and I had > to put the settings inside `server/etc/jetty.xml` to make it work. > Have I missed anything there? Seems like whatever I put in > `solr.in.cmd` is ignored or something. The weird thing is that I have tried > to use the options when starting Solr too and they didn't seem to work: > > solr start -c -Dsolr.ssl.checkPeerName=false > -Djavax.net.ssl.keyStorePassword=password > -Djavax.net.ssl.trustStorePassword=password > > 6) If I want my client to use its own certificate that the Solr server > will validate before executing any queries, can I do that? I see the > examples here: > https://cwiki.apache.org/confluence/display/solr/Enabling+SSL > where you seem to put cUrl use the server's certificate as a client > certificate (that's what I understand at least). Shouldn't the client > have its own certificate that the server should trust? And, also, I > have tried using the client without any client certificates and the server > accepts him. > Why do you feed cUrl with a certificate if that is not needed and is > there a way to make Solr validate the client certificate with a > whitelist or something similar? > > 7) The section `Run SolrCloud with SSL` here: > https://cwiki.apache.org/confluence/display/solr/Enabling+SSL#Enabling > SSL-Ru nSolrCloudwithSSL , seems weird to me because I don't see what > it has to do with SSL (maybe apart from the > `-Dsolr.ssl.checkPeerName=false` option). > Why does it make two nodes there? Do you have to make two nodes to > setup SSL on cloud mode?!?! As far as I see the -s option has nothing > to do with SSL either. So what does this section demonstrate? I am > already running Solr in cloud mode (using the -c option) with SSL > enabled and I didn't have to do any of those! > > > > > > That was most of it. J > > Thanks for your patience and any feedback would be welcome (I hope I > have not missed too much and I am totally out of topic), > > Kostas > > > >
