Re: New to Solr 5.5
So I was able to create the core using the cli not with root but with 'solr' user access. I sudo as solr in the box and ran the command to create the core [solr@solr bin]$ ./solr create -c demo Copying configuration to new core instance directory: /var/solr/data/demo Creating new core 'demo' using command:http://localhost:8983/solr/admin/cores?action=CREATE=demo=demo { "responseHeader":{ "status":0, "QTime":2724}, "core":"demo"} This created the required file/folder structure under "/var/solr/data" It has all the default files, I need some direction to configure existing Cassandra keyspace into my solr core. I have already placed the Cassandra jdbc drivers into my Java/lib, I believe I should be using the 'DIH' example configs for getting the data into solr ? Regards, Bhanu Prasad On Tue, Mar 15, 2016 at 11:38 AM, Erick Ericksonwrote: > Creating a core in stand-alone mode _also_ requires that the config files > are findable, in this case on disk rather than on Zookeeper > (sorry for the misdirection). > > So you need to create a directory, usually under solr_home that has > the conf directory in it. That is the "instanceDir" that is one of the > core creation > parameters. > > Yes, this is a little arcane > > Best, > Erick > > On Tue, Mar 15, 2016 at 7:34 AM, Bhanu Prasad > wrote: > > I went through the solr-ref-guide. I got a brief idea on how it works, > But > > I can't help to think as to why I am unable to create a core through web > UI > > ? Does it have any dependency on SolrCloud ? If I am planning to run an > > standalone instance do I need to create core's only through the command > > line and with 'solr' user privileges ? Please advise. > > > > > > > > On Mon, Mar 14, 2016 at 6:26 PM, Erick Erickson > > > wrote: > > > >> OK, take Cassandra out of it for the time being and spend > >> some time familiarizing yourself with Solr would be my > >> advice ;) > >> > >> Yeah, the Solr documentation is a bit scattered, but your most > >> complete and up to date reference is the Solr reference guide > >> here: > >> > >> In particular, see the > >> > >> > https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference > >> upconfig command. > >> > >> You can download the complete reference guide through the link in the > >> upper left. > >> > >> The general idea here is that your configurations (solrconfig.xml, > >> schema.xml and all the rest) are > >> kept in Zookeeper. When creating a collection, you must reference that > >> configuration set. The > >> examples automatically push the configuration set up to Zookeeper, > >> which you can see in the > >> adminUI>>cloud>>tree view. > >> > >> Best, > >> Erick > >> > >> On Mon, Mar 14, 2016 at 12:20 PM, Bhanu Prasad > >> wrote: > >> > Hi Erick, > >> > > >> > I am very new to this, I haven't uploaded any configsets. I need help > to > >> > get existing cassandra keyspace into solr to do analysis. I am > completely > >> > new to this technology so having trouble with finding right > documentation > >> > on how to do it. > >> > > >> > Regards, > >> > Bhanu > >> > > >> > On Mon, Mar 14, 2016 at 3:11 PM, Erick Erickson < > erickerick...@gmail.com > >> > > >> > wrote: > >> > > >> >> What configsets have you uploaded to Solr? The canned example does > >> >> this for you. The configurations must reside in Zookeeper, NOT on the > >> >> local disk. I think that's probably what you're seeing... > >> >> > >> >> Best, > >> >> Erick > >> >> > >> >> On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasad < > bhanupras...@gmail.com> > >> >> wrote: > >> >> > I was able to create a core using create -c option, But this time > with > >> >> user > >> >> > as solr. It worked. How can I make sure that the solr user is > running > >> the > >> >> > webapplication command requests as well ? Any help ? > >> >> > > >> >> > [solr@solr bin]$ ./solr create -c cassie > >> >> > > >> >> > Copying configuration to new core instance directory: > >> >> > /var/solr/data/cassie > >> >> > > >> >> > Creating new core 'cassie' using command: > >> >> > > >> >> > >> > http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie > >> >> > > >> >> > { > >> >> > "responseHeader":{ > >> >> > "status":0, > >> >> > "QTime":709}, > >> >> > "core":"cassie"} > >> >> > > >> >> > Regards, > >> >> > Bhanu Prasad > >> >> > > >> >> > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad < > bhanupras...@gmail.com > >> > > >> >> > wrote: > >> >> > > >> >> >> Hello, > >> >> >> > >> >> >> > >> >> >> I installed a new solr instance in lab on Cent OS 7 > >> >> >> > >> >> >> # java -version > >> >> >> java version "1.8.0_72" > >> >> >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) > >> >> >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) > >> >> >> > >> >> >> #wget > >> http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz > >> >> >> #tar -zxvf
Re: New to Solr 5.5
Creating a core in stand-alone mode _also_ requires that the config files are findable, in this case on disk rather than on Zookeeper (sorry for the misdirection). So you need to create a directory, usually under solr_home that has the conf directory in it. That is the "instanceDir" that is one of the core creation parameters. Yes, this is a little arcane Best, Erick On Tue, Mar 15, 2016 at 7:34 AM, Bhanu Prasadwrote: > I went through the solr-ref-guide. I got a brief idea on how it works, But > I can't help to think as to why I am unable to create a core through web UI > ? Does it have any dependency on SolrCloud ? If I am planning to run an > standalone instance do I need to create core's only through the command > line and with 'solr' user privileges ? Please advise. > > > > On Mon, Mar 14, 2016 at 6:26 PM, Erick Erickson > wrote: > >> OK, take Cassandra out of it for the time being and spend >> some time familiarizing yourself with Solr would be my >> advice ;) >> >> Yeah, the Solr documentation is a bit scattered, but your most >> complete and up to date reference is the Solr reference guide >> here: >> >> In particular, see the >> >> https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference >> upconfig command. >> >> You can download the complete reference guide through the link in the >> upper left. >> >> The general idea here is that your configurations (solrconfig.xml, >> schema.xml and all the rest) are >> kept in Zookeeper. When creating a collection, you must reference that >> configuration set. The >> examples automatically push the configuration set up to Zookeeper, >> which you can see in the >> adminUI>>cloud>>tree view. >> >> Best, >> Erick >> >> On Mon, Mar 14, 2016 at 12:20 PM, Bhanu Prasad >> wrote: >> > Hi Erick, >> > >> > I am very new to this, I haven't uploaded any configsets. I need help to >> > get existing cassandra keyspace into solr to do analysis. I am completely >> > new to this technology so having trouble with finding right documentation >> > on how to do it. >> > >> > Regards, >> > Bhanu >> > >> > On Mon, Mar 14, 2016 at 3:11 PM, Erick Erickson > > >> > wrote: >> > >> >> What configsets have you uploaded to Solr? The canned example does >> >> this for you. The configurations must reside in Zookeeper, NOT on the >> >> local disk. I think that's probably what you're seeing... >> >> >> >> Best, >> >> Erick >> >> >> >> On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasad >> >> wrote: >> >> > I was able to create a core using create -c option, But this time with >> >> user >> >> > as solr. It worked. How can I make sure that the solr user is running >> the >> >> > webapplication command requests as well ? Any help ? >> >> > >> >> > [solr@solr bin]$ ./solr create -c cassie >> >> > >> >> > Copying configuration to new core instance directory: >> >> > /var/solr/data/cassie >> >> > >> >> > Creating new core 'cassie' using command: >> >> > >> >> >> http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie >> >> > >> >> > { >> >> > "responseHeader":{ >> >> > "status":0, >> >> > "QTime":709}, >> >> > "core":"cassie"} >> >> > >> >> > Regards, >> >> > Bhanu Prasad >> >> > >> >> > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad > > >> >> > wrote: >> >> > >> >> >> Hello, >> >> >> >> >> >> >> >> >> I installed a new solr instance in lab on Cent OS 7 >> >> >> >> >> >> # java -version >> >> >> java version "1.8.0_72" >> >> >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) >> >> >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) >> >> >> >> >> >> #wget >> http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz >> >> >> #tar -zxvf solr-5.5.0.tgz >> >> >> #cd solr-5.5.0 >> >> >> >> >> >> #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz >> >> >> #id solr >> >> >> uid=1000(solr) gid=1000(solr) groups=1000(solr) >> >> >> >> >> >> I am getting an error when creating a new core from the UI and CLI. >> >> Kindly >> >> >> someone guide me what I am missing ? >> >> >> >> >> >> org.apache.solr.common.SolrException: Could not load conf for core >> >> netflow: Error loading solr config from >> >> /var/solr/data/netflow/conf/solrconfig.xml >> >> >> at >> >> >> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) >> >> >> at >> >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) >> >> >> at >> >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) >> >> >> at >> >> >> org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) >> >> >> at >> >> >> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) >> >> >> at >> >> >> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) >> >> >> at >> >> >>
Re: New to Solr 5.5
I went through the solr-ref-guide. I got a brief idea on how it works, But I can't help to think as to why I am unable to create a core through web UI ? Does it have any dependency on SolrCloud ? If I am planning to run an standalone instance do I need to create core's only through the command line and with 'solr' user privileges ? Please advise. On Mon, Mar 14, 2016 at 6:26 PM, Erick Ericksonwrote: > OK, take Cassandra out of it for the time being and spend > some time familiarizing yourself with Solr would be my > advice ;) > > Yeah, the Solr documentation is a bit scattered, but your most > complete and up to date reference is the Solr reference guide > here: > > In particular, see the > > https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference > upconfig command. > > You can download the complete reference guide through the link in the > upper left. > > The general idea here is that your configurations (solrconfig.xml, > schema.xml and all the rest) are > kept in Zookeeper. When creating a collection, you must reference that > configuration set. The > examples automatically push the configuration set up to Zookeeper, > which you can see in the > adminUI>>cloud>>tree view. > > Best, > Erick > > On Mon, Mar 14, 2016 at 12:20 PM, Bhanu Prasad > wrote: > > Hi Erick, > > > > I am very new to this, I haven't uploaded any configsets. I need help to > > get existing cassandra keyspace into solr to do analysis. I am completely > > new to this technology so having trouble with finding right documentation > > on how to do it. > > > > Regards, > > Bhanu > > > > On Mon, Mar 14, 2016 at 3:11 PM, Erick Erickson > > > wrote: > > > >> What configsets have you uploaded to Solr? The canned example does > >> this for you. The configurations must reside in Zookeeper, NOT on the > >> local disk. I think that's probably what you're seeing... > >> > >> Best, > >> Erick > >> > >> On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasad > >> wrote: > >> > I was able to create a core using create -c option, But this time with > >> user > >> > as solr. It worked. How can I make sure that the solr user is running > the > >> > webapplication command requests as well ? Any help ? > >> > > >> > [solr@solr bin]$ ./solr create -c cassie > >> > > >> > Copying configuration to new core instance directory: > >> > /var/solr/data/cassie > >> > > >> > Creating new core 'cassie' using command: > >> > > >> > http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie > >> > > >> > { > >> > "responseHeader":{ > >> > "status":0, > >> > "QTime":709}, > >> > "core":"cassie"} > >> > > >> > Regards, > >> > Bhanu Prasad > >> > > >> > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad > > >> > wrote: > >> > > >> >> Hello, > >> >> > >> >> > >> >> I installed a new solr instance in lab on Cent OS 7 > >> >> > >> >> # java -version > >> >> java version "1.8.0_72" > >> >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) > >> >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) > >> >> > >> >> #wget > http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz > >> >> #tar -zxvf solr-5.5.0.tgz > >> >> #cd solr-5.5.0 > >> >> > >> >> #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz > >> >> #id solr > >> >> uid=1000(solr) gid=1000(solr) groups=1000(solr) > >> >> > >> >> I am getting an error when creating a new core from the UI and CLI. > >> Kindly > >> >> someone guide me what I am missing ? > >> >> > >> >> org.apache.solr.common.SolrException: Could not load conf for core > >> netflow: Error loading solr config from > >> /var/solr/data/netflow/conf/solrconfig.xml > >> >> at > >> > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) > >> >> at > >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) > >> >> at > >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) > >> >> at > >> > org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) > >> >> at > >> > org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) > >> >> at > >> > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) > >> >> at > >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155) > >> >> at > >> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) > >> >> at > >> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439) > >> >> at > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) > >> >> at > >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183) > >> >> at > >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > >>
Re: New to Solr 5.5
OK, take Cassandra out of it for the time being and spend some time familiarizing yourself with Solr would be my advice ;) Yeah, the Solr documentation is a bit scattered, but your most complete and up to date reference is the Solr reference guide here: In particular, see the https://cwiki.apache.org/confluence/display/solr/Solr+Start+Script+Reference upconfig command. You can download the complete reference guide through the link in the upper left. The general idea here is that your configurations (solrconfig.xml, schema.xml and all the rest) are kept in Zookeeper. When creating a collection, you must reference that configuration set. The examples automatically push the configuration set up to Zookeeper, which you can see in the adminUI>>cloud>>tree view. Best, Erick On Mon, Mar 14, 2016 at 12:20 PM, Bhanu Prasadwrote: > Hi Erick, > > I am very new to this, I haven't uploaded any configsets. I need help to > get existing cassandra keyspace into solr to do analysis. I am completely > new to this technology so having trouble with finding right documentation > on how to do it. > > Regards, > Bhanu > > On Mon, Mar 14, 2016 at 3:11 PM, Erick Erickson > wrote: > >> What configsets have you uploaded to Solr? The canned example does >> this for you. The configurations must reside in Zookeeper, NOT on the >> local disk. I think that's probably what you're seeing... >> >> Best, >> Erick >> >> On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasad >> wrote: >> > I was able to create a core using create -c option, But this time with >> user >> > as solr. It worked. How can I make sure that the solr user is running the >> > webapplication command requests as well ? Any help ? >> > >> > [solr@solr bin]$ ./solr create -c cassie >> > >> > Copying configuration to new core instance directory: >> > /var/solr/data/cassie >> > >> > Creating new core 'cassie' using command: >> > >> http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie >> > >> > { >> > "responseHeader":{ >> > "status":0, >> > "QTime":709}, >> > "core":"cassie"} >> > >> > Regards, >> > Bhanu Prasad >> > >> > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad >> > wrote: >> > >> >> Hello, >> >> >> >> >> >> I installed a new solr instance in lab on Cent OS 7 >> >> >> >> # java -version >> >> java version "1.8.0_72" >> >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) >> >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) >> >> >> >> #wget http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz >> >> #tar -zxvf solr-5.5.0.tgz >> >> #cd solr-5.5.0 >> >> >> >> #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz >> >> #id solr >> >> uid=1000(solr) gid=1000(solr) groups=1000(solr) >> >> >> >> I am getting an error when creating a new core from the UI and CLI. >> Kindly >> >> someone guide me what I am missing ? >> >> >> >> org.apache.solr.common.SolrException: Could not load conf for core >> netflow: Error loading solr config from >> /var/solr/data/netflow/conf/solrconfig.xml >> >> at >> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) >> >> at >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) >> >> at >> org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) >> >> at >> org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) >> >> at >> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) >> >> at >> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) >> >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155) >> >> at >> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) >> >> at >> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439) >> >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) >> >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183) >> >> at >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >> >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >> >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >> >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >> >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >> >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >> >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >> >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >> >> at >>
Re: New to Solr 5.5
Hi Erick, I am very new to this, I haven't uploaded any configsets. I need help to get existing cassandra keyspace into solr to do analysis. I am completely new to this technology so having trouble with finding right documentation on how to do it. Regards, Bhanu On Mon, Mar 14, 2016 at 3:11 PM, Erick Ericksonwrote: > What configsets have you uploaded to Solr? The canned example does > this for you. The configurations must reside in Zookeeper, NOT on the > local disk. I think that's probably what you're seeing... > > Best, > Erick > > On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasad > wrote: > > I was able to create a core using create -c option, But this time with > user > > as solr. It worked. How can I make sure that the solr user is running the > > webapplication command requests as well ? Any help ? > > > > [solr@solr bin]$ ./solr create -c cassie > > > > Copying configuration to new core instance directory: > > /var/solr/data/cassie > > > > Creating new core 'cassie' using command: > > > http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie > > > > { > > "responseHeader":{ > > "status":0, > > "QTime":709}, > > "core":"cassie"} > > > > Regards, > > Bhanu Prasad > > > > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad > > wrote: > > > >> Hello, > >> > >> > >> I installed a new solr instance in lab on Cent OS 7 > >> > >> # java -version > >> java version "1.8.0_72" > >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) > >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) > >> > >> #wget http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz > >> #tar -zxvf solr-5.5.0.tgz > >> #cd solr-5.5.0 > >> > >> #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz > >> #id solr > >> uid=1000(solr) gid=1000(solr) groups=1000(solr) > >> > >> I am getting an error when creating a new core from the UI and CLI. > Kindly > >> someone guide me what I am missing ? > >> > >> org.apache.solr.common.SolrException: Could not load conf for core > netflow: Error loading solr config from > /var/solr/data/netflow/conf/solrconfig.xml > >> at > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) > >> at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) > >> at > org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) > >> at > org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) > >> at > org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) > >> at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) > >> at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155) > >> at > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) > >> at > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439) > >> at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) > >> at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183) > >> at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > >> at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > >> at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > >> at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > >> at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > >> at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > >> at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > >> at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > >> at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) > >> at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > >> at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > >> at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) > >> at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > >> at org.eclipse.jetty.server.Server.handle(Server.java:499) > >> at > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > >> at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) > >> at > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > >> at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > >> at >
Re: New to Solr 5.5
What configsets have you uploaded to Solr? The canned example does this for you. The configurations must reside in Zookeeper, NOT on the local disk. I think that's probably what you're seeing... Best, Erick On Mon, Mar 14, 2016 at 11:33 AM, Bhanu Prasadwrote: > I was able to create a core using create -c option, But this time with user > as solr. It worked. How can I make sure that the solr user is running the > webapplication command requests as well ? Any help ? > > [solr@solr bin]$ ./solr create -c cassie > > Copying configuration to new core instance directory: > /var/solr/data/cassie > > Creating new core 'cassie' using command: > http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie > > { > "responseHeader":{ > "status":0, > "QTime":709}, > "core":"cassie"} > > Regards, > Bhanu Prasad > > On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasad > wrote: > >> Hello, >> >> >> I installed a new solr instance in lab on Cent OS 7 >> >> # java -version >> java version "1.8.0_72" >> Java(TM) SE Runtime Environment (build 1.8.0_72-b15) >> Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) >> >> #wget http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz >> #tar -zxvf solr-5.5.0.tgz >> #cd solr-5.5.0 >> >> #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz >> #id solr >> uid=1000(solr) gid=1000(solr) groups=1000(solr) >> >> I am getting an error when creating a new core from the UI and CLI. Kindly >> someone guide me what I am missing ? >> >> org.apache.solr.common.SolrException: Could not load conf for core netflow: >> Error loading solr config from /var/solr/data/netflow/conf/solrconfig.xml >> at >> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) >> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) >> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) >> at >> org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) >> at >> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) >> at >> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) >> at >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155) >> at >> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) >> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) >> at >> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183) >> at >> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) >> at >> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) >> at >> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) >> at >> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) >> at >> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) >> at >> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) >> at >> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) >> at >> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) >> at >> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) >> at >> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) >> at >> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) >> at org.eclipse.jetty.server.Server.handle(Server.java:499) >> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) >> at >> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) >> at >> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) >> at >> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: org.apache.solr.common.SolrException: Error loading solr config >> from /var/solr/data/netflow/conf/solrconfig.xml >> at >> org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:174) >> at >> org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:98) >> at >> org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78) >> ...
Re: New to Solr 5.5
I was able to create a core using create -c option, But this time with user as solr. It worked. How can I make sure that the solr user is running the webapplication command requests as well ? Any help ? [solr@solr bin]$ ./solr create -c cassie Copying configuration to new core instance directory: /var/solr/data/cassie Creating new core 'cassie' using command: http://localhost:8983/solr/admin/cores?action=CREATE=cassie=cassie { "responseHeader":{ "status":0, "QTime":709}, "core":"cassie"} Regards, Bhanu Prasad On Mon, Mar 14, 2016 at 1:30 PM, Bhanu Prasadwrote: > Hello, > > > I installed a new solr instance in lab on Cent OS 7 > > # java -version > java version "1.8.0_72" > Java(TM) SE Runtime Environment (build 1.8.0_72-b15) > Java HotSpot(TM) 64-Bit Server VM (build 25.72-b15, mixed mode) > > #wget http://apache.mirror.gtcomm.net/lucene/solr/5.5.0/solr-5.5.0.tgz > #tar -zxvf solr-5.5.0.tgz > #cd solr-5.5.0 > > #bin/install_solr_service.sh /tmp/solr-5.5.0.tgz > #id solr > uid=1000(solr) gid=1000(solr) groups=1000(solr) > > I am getting an error when creating a new core from the UI and CLI. Kindly > someone guide me what I am missing ? > > org.apache.solr.common.SolrException: Could not load conf for core netflow: > Error loading solr config from /var/solr/data/netflow/conf/solrconfig.xml > at > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:84) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:812) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:751) > at > org.apache.solr.handler.admin.CoreAdminOperation$1.call(CoreAdminOperation.java:129) > at > org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:354) > at > org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:153) > at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155) > at > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:439) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:225) > at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:183) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) > at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) > at org.eclipse.jetty.server.Server.handle(Server.java:499) > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) > at > org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) > at > org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.solr.common.SolrException: Error loading solr config > from /var/solr/data/netflow/conf/solrconfig.xml > at > org.apache.solr.core.SolrConfig.readFromResourceLoader(SolrConfig.java:174) > at > org.apache.solr.core.ConfigSetService.createSolrConfig(ConfigSetService.java:98) > at > org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:78) > ... 30 more > Caused by: org.apache.solr.core.SolrResourceNotFoundException: Can't find > resource 'solrconfig.xml' in classpath or '/var/solr/data/netflow' > at > org.apache.solr.core.SolrResourceLoader.openResource(SolrResourceLoader.java:408) > at > org.apache.solr.core.SolrResourceLoader.openConfig(SolrResourceLoader.java:363) > at org.apache.solr.core.Config.(Config.java:120) > at
Re: New To Solr, getting error using the quick start guide
Hi Swawn. Thanks so much for the response. I will do so more tests on this and send more info in the next day or 2, maybe that will illuminate something. I hope you are having a great weekend. All the best. Will Ferrer On Sat, Mar 28, 2015 at 10:28 AM, Shawn Heisey apa...@elyograg.org wrote: On 3/27/2015 8:00 PM, Will ferrer wrote: I am new to solr and trying to run through the quick start guide ( http://lucene.apache.org/solr/quickstart.html). The installation seems fine but then I run: bin/solr start -e cloud -noprompt You are starting the cloud example with no prompts. http://localhost:8983/solr/#/ shows data in my web browser, but the cloud tab is empty under graph. Any advice any one give me to get me started here with the product would be very appreciated. The console log that you included with your message said nothing about creating the gettingstarted collection, but when I try the command you used on the following setups, it creates the collection every time: *) Linux, from the tags/lucene_solr_5_0_0 source. *) Windows 8.1, from the binary 5.0.0 download. *) Linux, from the branch_5x source. Here's my console log from the first item above - Solr built from the tags/lucene_solr_5_0_0 source: --- elyograg@sauron:~/asf/lucene_solr_5_0_0/solr$ bin/solr -e cloud -noprompt Welcome to the SolrCloud example! Starting up 2 Solr nodes for your example SolrCloud cluster. Creating Solr home directory /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node1/solr Cloning Solr home directory /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node1 into /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node2 Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -s example/cloud/node1/solr -p 8983 Waiting to see Solr listening on port 8983 [/] Started Solr server on port 8983 (pid=13260). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983 Waiting to see Solr listening on port 7574 [/] Started Solr server on port 7574 (pid=13419). Happy searching! Connecting to ZooKeeper at localhost:9983 Uploading /home/elyograg/asf/lucene_solr_5_0_0/solr/server/solr/configsets/data_driven_schema_configs/conf for config gettingstarted to ZooKeeper at localhost:9983 Creating new collection 'gettingstarted' using command: http://166.70.79.221:7574/solr/admin/collections?action=CREATEname=gettingstartednumShards=2replicationFactor=2maxShardsPerNode=2collection.configName=gettingstarted { responseHeader:{ status:0, QTime:6869}, success:{:{ responseHeader:{ status:0, QTime:6387}, core:gettingstarted_shard1_replica2}}} SolrCloud example running, please visit http://localhost:8983/solr elyograg@sauron:~/asf/lucene_solr_5_0_0/solr$ --- I don't know why it's not creating the collection for you, unless maybe you are running a different version built from older source code or something. Thanks, Shawn
Re: New To Solr, getting error using the quick start guide
On 3/27/2015 8:00 PM, Will ferrer wrote: I am new to solr and trying to run through the quick start guide ( http://lucene.apache.org/solr/quickstart.html). The installation seems fine but then I run: bin/solr start -e cloud -noprompt You are starting the cloud example with no prompts. http://localhost:8983/solr/#/ shows data in my web browser, but the cloud tab is empty under graph. Any advice any one give me to get me started here with the product would be very appreciated. The console log that you included with your message said nothing about creating the gettingstarted collection, but when I try the command you used on the following setups, it creates the collection every time: *) Linux, from the tags/lucene_solr_5_0_0 source. *) Windows 8.1, from the binary 5.0.0 download. *) Linux, from the branch_5x source. Here's my console log from the first item above - Solr built from the tags/lucene_solr_5_0_0 source: --- elyograg@sauron:~/asf/lucene_solr_5_0_0/solr$ bin/solr -e cloud -noprompt Welcome to the SolrCloud example! Starting up 2 Solr nodes for your example SolrCloud cluster. Creating Solr home directory /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node1/solr Cloning Solr home directory /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node1 into /home/elyograg/asf/lucene_solr_5_0_0/solr/example/cloud/node2 Starting up SolrCloud node1 on port 8983 using command: solr start -cloud -s example/cloud/node1/solr -p 8983 Waiting to see Solr listening on port 8983 [/] Started Solr server on port 8983 (pid=13260). Happy searching! Starting node2 on port 7574 using command: solr start -cloud -s example/cloud/node2/solr -p 7574 -z localhost:9983 Waiting to see Solr listening on port 7574 [/] Started Solr server on port 7574 (pid=13419). Happy searching! Connecting to ZooKeeper at localhost:9983 Uploading /home/elyograg/asf/lucene_solr_5_0_0/solr/server/solr/configsets/data_driven_schema_configs/conf for config gettingstarted to ZooKeeper at localhost:9983 Creating new collection 'gettingstarted' using command: http://166.70.79.221:7574/solr/admin/collections?action=CREATEname=gettingstartednumShards=2replicationFactor=2maxShardsPerNode=2collection.configName=gettingstarted { responseHeader:{ status:0, QTime:6869}, success:{:{ responseHeader:{ status:0, QTime:6387}, core:gettingstarted_shard1_replica2}}} SolrCloud example running, please visit http://localhost:8983/solr elyograg@sauron:~/asf/lucene_solr_5_0_0/solr$ --- I don't know why it's not creating the collection for you, unless maybe you are running a different version built from older source code or something. Thanks, Shawn
Re: New to Solr can someone help me to know if Solr fits my use case
Thanks a lot Alexandre for the response much appreciated. Thanks Saurabh On Fri, Mar 28, 2014 at 8:56 AM, Alexandre Rafalovitch arafa...@gmail.com wrote: 1. You don't actually put PDF/Word into Solr. Instead, it is run through content and metadata extraction process and then index that. This is important because a computer does not understand what you are looking for when you open a PDF. It only understand whatever text is possible to extract. In case of PDF it is often not much at all, unless it was generated with accessibility layer in place. You can experiment with what you can extract by downloading a standalone Apache Tika install, which has a command line version or using Solr's extractOnly flag. Solr, internally, uses Tika, so the results should be the same. 2) When you do a search you can do field:(Keyword1 Keyword2 Keyword3 Keyword4) and you get as results any document that matches one of those. Not sure about 1000 of them in one go, but certainly a large number. On the other hand, if you have same keywords all the time and you are trying to match documents against them, you might be more interested in Elastic Search's percolator (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html ) or in Luwak (https://github.com/flaxsearch/luwak). Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Fri, Mar 28, 2014 at 10:05 AM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Thanks a lot Alex for your reply, Appreciate the same. So if i leave the line no part. 1. I guess putting pdf/word in solr for search can be done, These documents will go go in solr. 2. For search any automatic way to give a excel sheet or large search keywords to search for . ie i have 1000's of words that i want to search in doc can i do it collectively or send search queries one by one. Thanks Saurabh On Fri, Mar 28, 2014 at 6:48 AM, Alexandre Rafalovitch arafa...@gmail.com wrote: This feels somewhat backwards. It's very hard to extract Line-Number information out of MSWord and next to impossible from PDF. So, it's not whether the Solr is a good fit or not here is that maybe your whole architecture has a major issue. Can you do this/what you want by hand at least once? Down to the precision you want? If you can, then yes you probably can automate the searching with Solr, though you will still have serious issues (sentence crossing line-boundaries, etc). But I suspect your whole approach will change once you try to do this manually. Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Thu, Mar 27, 2014 at 11:46 PM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Can anyone help me please. Hi All, I am new to Solr and from initial reading i am quite convinced Solr will be of great help. Can anyone help in making that decision. Usecase: 1. I will have PDF,Word docs generated daily/weekly ( lot of them ) which kinds of get overwritten frequently. 2. I have a dictionary kind of thing ( having a list of which words/small sentences should be part of above docs , words which cannot be and alternatives for some ). 3. Now i want Solr to search my Docs produced in step 1 to be searched for words/small sentences from step 2 and give me my Doc Name/line no in which they exist. Will Solr be a good help to me, If anybody can help giving some examples that will be great. Appreciate your help and patience. Thanks Saurabh
Re: New to Solr can someone help me to know if Solr fits my use case
Can anyone help me please. Hi All, I am new to Solr and from initial reading i am quite convinced Solr will be of great help. Can anyone help in making that decision. Usecase: 1. I will have PDF,Word docs generated daily/weekly ( lot of them ) which kinds of get overwritten frequently. 2. I have a dictionary kind of thing ( having a list of which words/small sentences should be part of above docs , words which cannot be and alternatives for some ). 3. Now i want Solr to search my Docs produced in step 1 to be searched for words/small sentences from step 2 and give me my Doc Name/line no in which they exist. Will Solr be a good help to me, If anybody can help giving some examples that will be great. Appreciate your help and patience. Thanks Saurabh
Re: New to Solr can someone help me to know if Solr fits my use case
This feels somewhat backwards. It's very hard to extract Line-Number information out of MSWord and next to impossible from PDF. So, it's not whether the Solr is a good fit or not here is that maybe your whole architecture has a major issue. Can you do this/what you want by hand at least once? Down to the precision you want? If you can, then yes you probably can automate the searching with Solr, though you will still have serious issues (sentence crossing line-boundaries, etc). But I suspect your whole approach will change once you try to do this manually. Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Thu, Mar 27, 2014 at 11:46 PM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Can anyone help me please. Hi All, I am new to Solr and from initial reading i am quite convinced Solr will be of great help. Can anyone help in making that decision. Usecase: 1. I will have PDF,Word docs generated daily/weekly ( lot of them ) which kinds of get overwritten frequently. 2. I have a dictionary kind of thing ( having a list of which words/small sentences should be part of above docs , words which cannot be and alternatives for some ). 3. Now i want Solr to search my Docs produced in step 1 to be searched for words/small sentences from step 2 and give me my Doc Name/line no in which they exist. Will Solr be a good help to me, If anybody can help giving some examples that will be great. Appreciate your help and patience. Thanks Saurabh
Re: New to Solr can someone help me to know if Solr fits my use case
Thanks a lot Alex for your reply, Appreciate the same. So if i leave the line no part. 1. I guess putting pdf/word in solr for search can be done, These documents will go go in solr. 2. For search any automatic way to give a excel sheet or large search keywords to search for . ie i have 1000's of words that i want to search in doc can i do it collectively or send search queries one by one. Thanks Saurabh On Fri, Mar 28, 2014 at 6:48 AM, Alexandre Rafalovitch arafa...@gmail.com wrote: This feels somewhat backwards. It's very hard to extract Line-Number information out of MSWord and next to impossible from PDF. So, it's not whether the Solr is a good fit or not here is that maybe your whole architecture has a major issue. Can you do this/what you want by hand at least once? Down to the precision you want? If you can, then yes you probably can automate the searching with Solr, though you will still have serious issues (sentence crossing line-boundaries, etc). But I suspect your whole approach will change once you try to do this manually. Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Thu, Mar 27, 2014 at 11:46 PM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Can anyone help me please. Hi All, I am new to Solr and from initial reading i am quite convinced Solr will be of great help. Can anyone help in making that decision. Usecase: 1. I will have PDF,Word docs generated daily/weekly ( lot of them ) which kinds of get overwritten frequently. 2. I have a dictionary kind of thing ( having a list of which words/small sentences should be part of above docs , words which cannot be and alternatives for some ). 3. Now i want Solr to search my Docs produced in step 1 to be searched for words/small sentences from step 2 and give me my Doc Name/line no in which they exist. Will Solr be a good help to me, If anybody can help giving some examples that will be great. Appreciate your help and patience. Thanks Saurabh
Re: New to Solr can someone help me to know if Solr fits my use case
1. You don't actually put PDF/Word into Solr. Instead, it is run through content and metadata extraction process and then index that. This is important because a computer does not understand what you are looking for when you open a PDF. It only understand whatever text is possible to extract. In case of PDF it is often not much at all, unless it was generated with accessibility layer in place. You can experiment with what you can extract by downloading a standalone Apache Tika install, which has a command line version or using Solr's extractOnly flag. Solr, internally, uses Tika, so the results should be the same. 2) When you do a search you can do field:(Keyword1 Keyword2 Keyword3 Keyword4) and you get as results any document that matches one of those. Not sure about 1000 of them in one go, but certainly a large number. On the other hand, if you have same keywords all the time and you are trying to match documents against them, you might be more interested in Elastic Search's percolator (http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-percolate.html ) or in Luwak (https://github.com/flaxsearch/luwak). Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Fri, Mar 28, 2014 at 10:05 AM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Thanks a lot Alex for your reply, Appreciate the same. So if i leave the line no part. 1. I guess putting pdf/word in solr for search can be done, These documents will go go in solr. 2. For search any automatic way to give a excel sheet or large search keywords to search for . ie i have 1000's of words that i want to search in doc can i do it collectively or send search queries one by one. Thanks Saurabh On Fri, Mar 28, 2014 at 6:48 AM, Alexandre Rafalovitch arafa...@gmail.com wrote: This feels somewhat backwards. It's very hard to extract Line-Number information out of MSWord and next to impossible from PDF. So, it's not whether the Solr is a good fit or not here is that maybe your whole architecture has a major issue. Can you do this/what you want by hand at least once? Down to the precision you want? If you can, then yes you probably can automate the searching with Solr, though you will still have serious issues (sentence crossing line-boundaries, etc). But I suspect your whole approach will change once you try to do this manually. Regards, Alex. Personal website: http://www.outerthoughts.com/ Current project: http://www.solr-start.com/ - Accelerating your Solr proficiency On Thu, Mar 27, 2014 at 11:46 PM, Saurabh Agarwal sagarwal1...@gmail.com wrote: Can anyone help me please. Hi All, I am new to Solr and from initial reading i am quite convinced Solr will be of great help. Can anyone help in making that decision. Usecase: 1. I will have PDF,Word docs generated daily/weekly ( lot of them ) which kinds of get overwritten frequently. 2. I have a dictionary kind of thing ( having a list of which words/small sentences should be part of above docs , words which cannot be and alternatives for some ). 3. Now i want Solr to search my Docs produced in step 1 to be searched for words/small sentences from step 2 and give me my Doc Name/line no in which they exist. Will Solr be a good help to me, If anybody can help giving some examples that will be great. Appreciate your help and patience. Thanks Saurabh
Re: New to Solr - Need advice on clustering
On 26 November 2013 01:44, Anders Kåre Olsen a...@mail.dk wrote: Hi Solr-users I’m trying to setup Solr for search and indexing on the project I’m working on. My project is a e-commerce B2B solution. We are planning on setting up 2 frontend servers for the website, and I was planning on installing Solr on these servers. We are using Windows Server 2012 for the frontend servers. We are not expecting a huge load on the servers, so we expect these 2 servers to be adequate to handle both the website and search index. I have been looking at SolrCloud and ZooKeeper. Howver I have read that you need at least 3 ZooKeepers in an ensamble, and I only have 2 servers. I need to handle the situation where one of the servers crashes, so I need both servers to have a Solr index. [...] If you do not want to get into SolrCloud, a simpler solution might be a HTTP load balancer in front of the two Solr instances. Hardware load balancers are better, but more expensive. A software load balancer like haproxy should meet your needs. Regards, Gora
Re: New to Solr - Need advice on clustering
Hi Gora Thank you for your reply. We are planning on having a loadbalancer in front of our frontend servers. If I have two distinct solr indexes, how will I keep them synchronized? I expect that one of the frontend servers will have the task of updating the product repository on the e-commerce site. This server will then update the local solr index after product update has finished. Is there an easy way that I can keep the two indexes synchronized without solrcloud? Regards Anders -Oprindelig meddelelse- From: Gora Mohanty Sent: Tuesday, November 26, 2013 2:37 AM To: solr-user@lucene.apache.org Subject: Re: New to Solr - Need advice on clustering On 26 November 2013 01:44, Anders Kåre Olsen a...@mail.dk wrote: Hi Solr-users I’m trying to setup Solr for search and indexing on the project I’m working on. My project is a e-commerce B2B solution. We are planning on setting up 2 frontend servers for the website, and I was planning on installing Solr on these servers. We are using Windows Server 2012 for the frontend servers. We are not expecting a huge load on the servers, so we expect these 2 servers to be adequate to handle both the website and search index. I have been looking at SolrCloud and ZooKeeper. Howver I have read that you need at least 3 ZooKeepers in an ensamble, and I only have 2 servers. I need to handle the situation where one of the servers crashes, so I need both servers to have a Solr index. [...] If you do not want to get into SolrCloud, a simpler solution might be a HTTP load balancer in front of the two Solr instances. Hardware load balancers are better, but more expensive. A software load balancer like haproxy should meet your needs. Regards, Gora
Re: New to Solr - Need advice on clustering
Anders, Take a look at Solr Replication. Essentially, you'll treat one as a master one as a slave. Both master slave can be used to serve traffic. If one of them goes down, the other can be used as a master for the interim. http://wiki.apache.org/solr/SolrReplication Sameer. -- http://measuredsearch.com On Mon, Nov 25, 2013 at 9:50 PM, Anders Kåre Olsen a...@mail.dk wrote: Hi Gora Thank you for your reply. We are planning on having a loadbalancer in front of our frontend servers. If I have two distinct solr indexes, how will I keep them synchronized? I expect that one of the frontend servers will have the task of updating the product repository on the e-commerce site. This server will then update the local solr index after product update has finished. Is there an easy way that I can keep the two indexes synchronized without solrcloud? Regards Anders -Oprindelig meddelelse- From: Gora Mohanty Sent: Tuesday, November 26, 2013 2:37 AM To: solr-user@lucene.apache.org Subject: Re: New to Solr - Need advice on clustering On 26 November 2013 01:44, Anders Kåre Olsen a...@mail.dk wrote: Hi Solr-users I’m trying to setup Solr for search and indexing on the project I’m working on. My project is a e-commerce B2B solution. We are planning on setting up 2 frontend servers for the website, and I was planning on installing Solr on these servers. We are using Windows Server 2012 for the frontend servers. We are not expecting a huge load on the servers, so we expect these 2 servers to be adequate to handle both the website and search index. I have been looking at SolrCloud and ZooKeeper. Howver I have read that you need at least 3 ZooKeepers in an ensamble, and I only have 2 servers. I need to handle the situation where one of the servers crashes, so I need both servers to have a Solr index. [...] If you do not want to get into SolrCloud, a simpler solution might be a HTTP load balancer in front of the two Solr instances. Hardware load balancers are better, but more expensive. A software load balancer like haproxy should meet your needs. Regards, Gora
RE: New to Solr/Lucene design question
A third option - Use dynamic fields. Add a dynamic field call *_stash. This will allow new fields for documents to be added down the road without changing schema.xml, yet still allow you to query on fields like arresteeFirstName_stash without extra overhead. -Todd Feak -Original Message- From: Yogesh Chawla - PD [mailto:premiergenerat...@yahoo.com] Sent: Tuesday, January 20, 2009 2:30 PM To: solr-user@lucene.apache.org Subject: New to Solr/Lucene design question Hello All, We are using SOLR/Lucene as the search engine for an application we are designing. The application is a workflow application that can receive different types of documents. For example, we are currently working on getting booking documents but will also accept arrest documents later this year. We have defined a custom schema that incorporates some schemas designed by federal consortiums. From those schemas we pluck out values that we want SOLR/Lucene to index and search on and we go from our instance document to a SOLR document. The fields in our schema.xml look like this: fields !-- record-uri, unique identifier for any type of record -- field name=record-uri type=string indexed=true stored=true required=true / !-- stash-filepath, path to the entire XML document on the file system -- field name=stash-filepath type=string indexed=true stored=true required=true / !-- stash-content THIS IS THE FIELD I HAVE QUESTIONS ABOUT-- field name=stash-content type=string indexed=true stored=true termVectors=true multiValued=true ssomitNorms=true/ /fields Above, there is a field called stash-content. The goal is to take any search able data from any document type and put it in this field. For example, we would store data like this in XML format: add doc field name=stash-contentarrestee_firstname_Yogesh/field field name=stash-contentarrestee_lastname_Chawla/field field name=stash-contentarrestee_middlename_myMiddleName/field /doc /add The advantage to such an approach is that we can add new document types to search on and as long as they use the same semantics such as arrestee_firstname that we won't to update any code. It also makes the code simple and generic for any document type. We can search on first name like this for a starts with query:arrestee_firstname_Y*. We had to use the _ instead of a space so that each word would not be searched when a query was performed and only a single string would be searched. (hope that makes sense). The cons could be a performance hit. The other approach is to add fields explicitly like this: add doc field name=arrestee_firstnameYogesh/field field name=arrestee_lastnameChawla/field field name=arrestee_middlenamemyMiddleName/field /doc /add This approach seems more traditional. The pros of it are that it is straight forward. The cons are that every time we add a new document type to search on, we have to update schema.xml and the java code that creates SOLR documents. The number of documents that we will eventually want to search on is about 5 million. However, this will take a while to ramp up to and we are more immediately looking at searching on about 100,000. I am new to SOLR and just inherited this project with approach number 1. Is this something that is going to bite us in the future? Thanks, Yogesh
Re: New to Solr/Lucene design question
Hi Todd, I think I see what you are saying here. In our schema.xml we can define it like this: fields !-- record-uri, unique identifier for any type of record -- field name=record-uri type=string indexed=true stored=true required=true / !-- stash-filepath, path to the entire XML document on the file system -- field name=stash-filepath type=string indexed=true stored=true required=true / dynamicField name=*_stashString type=string indexed=true stored=true/ /fields and then add data like this: add doc field name=arrestee_firstname_stashStringYogesh/field field name=arrestee_lastname_stashStringChawla/field field name=arrestee_middlename_stashStringmyMiddleName/field /doc /add If we need to add other types of dynamic data types, we can do that at a later time by adding a different type of dynamic field. This way we are not querying a single field 'stash-content' but rather just the fields we are interested in and there is no need to change the java code or the schema.xml. Are we on the same wave length here? Thanks a lot for the suggestion, Yogesh - Original Message From: Feak, Todd todd.f...@smss.sony.com To: solr-user@lucene.apache.org Sent: Tuesday, January 20, 2009 4:49:56 PM Subject: RE: New to Solr/Lucene design question A third option - Use dynamic fields. Add a dynamic field call *_stash. This will allow new fields for documents to be added down the road without changing schema.xml, yet still allow you to query on fields like arresteeFirstName_stash without extra overhead. -Todd Feak -Original Message- From: Yogesh Chawla - PD [mailto:premiergenerat...@yahoo.com] Sent: Tuesday, January 20, 2009 2:30 PM To: solr-user@lucene.apache.org Subject: New to Solr/Lucene design question Hello All, We are using SOLR/Lucene as the search engine for an application we are designing. The application is a workflow application that can receive different types of documents. For example, we are currently working on getting booking documents but will also accept arrest documents later this year. We have defined a custom schema that incorporates some schemas designed by federal consortiums. From those schemas we pluck out values that we want SOLR/Lucene to index and search on and we go from our instance document to a SOLR document. The fields in our schema.xml look like this: fields !-- record-uri, unique identifier for any type of record -- field name=record-uri type=string indexed=true stored=true required=true / !-- stash-filepath, path to the entire XML document on the file system -- field name=stash-filepath type=string indexed=true stored=true required=true / !-- stash-content THIS IS THE FIELD I HAVE QUESTIONS ABOUT-- field name=stash-content type=string indexed=true stored=true termVectors=true multiValued=true ssomitNorms=true/ /fields Above, there is a field called stash-content. The goal is to take any search able data from any document type and put it in this field. For example, we would store data like this in XML format: add doc field name=stash-contentarrestee_firstname_Yogesh/field field name=stash-contentarrestee_lastname_Chawla/field field name=stash-contentarrestee_middlename_myMiddleName/field /doc /add The advantage to such an approach is that we can add new document types to search on and as long as they use the same semantics such as arrestee_firstname that we won't to update any code. It also makes the code simple and generic for any document type. We can search on first name like this for a starts with query:arrestee_firstname_Y*. We had to use the _ instead of a space so that each word would not be searched when a query was performed and only a single string would be searched. (hope that makes sense). The cons could be a performance hit. The other approach is to add fields explicitly like this: add doc field name=arrestee_firstnameYogesh/field field name=arrestee_lastnameChawla/field field name=arrestee_middlenamemyMiddleName/field /doc /add This approach seems more traditional. The pros of it are that it is straight forward. The cons are that every time we add a new document type to search on, we have to update schema.xml and the java code that creates SOLR documents. The number of documents that we will eventually want to search on is about 5 million. However, this will take a while to ramp up to and we are more immediately looking at searching on about 100,000. I am new to SOLR and just inherited this project with approach number 1. Is this something that is going to bite us in the future? Thanks, Yogesh
RE: New to Solr/Lucene design question
Yes, that's what I was suggesting. :) Might have to be careful with the extra underscore _ characters. Not sure if those will cause issue with dynamic fields. -Todd Feak -Original Message- From: Yogesh Chawla - PD [mailto:premiergenerat...@yahoo.com] Sent: Tuesday, January 20, 2009 3:14 PM To: solr-user@lucene.apache.org Subject: Re: New to Solr/Lucene design question Hi Todd, I think I see what you are saying here. In our schema.xml we can define it like this: fields !-- record-uri, unique identifier for any type of record -- field name=record-uri type=string indexed=true stored=true required=true / !-- stash-filepath, path to the entire XML document on the file system -- field name=stash-filepath type=string indexed=true stored=true required=true / dynamicField name=*_stashString type=string indexed=true stored=true/ /fields and then add data like this: add doc field name=arrestee_firstname_stashStringYogesh/field field name=arrestee_lastname_stashStringChawla/field field name=arrestee_middlename_stashStringmyMiddleName/field /doc /add If we need to add other types of dynamic data types, we can do that at a later time by adding a different type of dynamic field. This way we are not querying a single field 'stash-content' but rather just the fields we are interested in and there is no need to change the java code or the schema.xml. Are we on the same wave length here? Thanks a lot for the suggestion, Yogesh - Original Message From: Feak, Todd todd.f...@smss.sony.com To: solr-user@lucene.apache.org Sent: Tuesday, January 20, 2009 4:49:56 PM Subject: RE: New to Solr/Lucene design question A third option - Use dynamic fields. Add a dynamic field call *_stash. This will allow new fields for documents to be added down the road without changing schema.xml, yet still allow you to query on fields like arresteeFirstName_stash without extra overhead. -Todd Feak -Original Message- From: Yogesh Chawla - PD [mailto:premiergenerat...@yahoo.com] Sent: Tuesday, January 20, 2009 2:30 PM To: solr-user@lucene.apache.org Subject: New to Solr/Lucene design question Hello All, We are using SOLR/Lucene as the search engine for an application we are designing. The application is a workflow application that can receive different types of documents. For example, we are currently working on getting booking documents but will also accept arrest documents later this year. We have defined a custom schema that incorporates some schemas designed by federal consortiums. From those schemas we pluck out values that we want SOLR/Lucene to index and search on and we go from our instance document to a SOLR document. The fields in our schema.xml look like this: fields !-- record-uri, unique identifier for any type of record -- field name=record-uri type=string indexed=true stored=true required=true / !-- stash-filepath, path to the entire XML document on the file system -- field name=stash-filepath type=string indexed=true stored=true required=true / !-- stash-content THIS IS THE FIELD I HAVE QUESTIONS ABOUT-- field name=stash-content type=string indexed=true stored=true termVectors=true multiValued=true ssomitNorms=true/ /fields Above, there is a field called stash-content. The goal is to take any search able data from any document type and put it in this field. For example, we would store data like this in XML format: add doc field name=stash-contentarrestee_firstname_Yogesh/field field name=stash-contentarrestee_lastname_Chawla/field field name=stash-contentarrestee_middlename_myMiddleName/field /doc /add The advantage to such an approach is that we can add new document types to search on and as long as they use the same semantics such as arrestee_firstname that we won't to update any code. It also makes the code simple and generic for any document type. We can search on first name like this for a starts with query:arrestee_firstname_Y*. We had to use the _ instead of a space so that each word would not be searched when a query was performed and only a single string would be searched. (hope that makes sense). The cons could be a performance hit. The other approach is to add fields explicitly like this: add doc field name=arrestee_firstnameYogesh/field field name=arrestee_lastnameChawla/field field name=arrestee_middlenamemyMiddleName/field /doc /add This approach seems more traditional. The pros of it are that it is straight forward. The cons are that every time we add a new document type to search on, we have to update schema.xml and the java code that creates SOLR documents. The number of documents that we will eventually want to search on is about 5 million. However, this will take a while to ramp up to and we are more immediately looking at searching on about 100,000. I am new to SOLR and just inherited this project with approach number 1. Is this something
Re: new to solr
Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Hello, I am new to solr. Welcome! I followed solr online tutorial to get the example work. The search result is xml. I wonder if there is a way to show result in a form. I saw there is example.xsl in conf/xslt directory. I really don't know how to do it. Anyone has some ideas for me. I really appreciate it! Are you asking how to display results for people to see? A nicely formatted website? Solr (a database) does not aim to solve the display side... but there are lots of clients to help integrate with your website. php/java/.net/ruby/etc ryan
RE: new to solr
Thanks so much for your reply! Please tell me what example.xsl is for in conf/xslt. Please let me know where the search result is located. I can use php or .net to display the result in web. Is it created on fly? Thanks, Xiaohui -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Monday, January 14, 2008 11:37 AM To: solr-user@lucene.apache.org Subject: Re: new to solr Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Hello, I am new to solr. Welcome! I followed solr online tutorial to get the example work. The search result is xml. I wonder if there is a way to show result in a form. I saw there is example.xsl in conf/xslt directory. I really don't know how to do it. Anyone has some ideas for me. I really appreciate it! Are you asking how to display results for people to see? A nicely formatted website? Solr (a database) does not aim to solve the display side... but there are lots of clients to help integrate with your website. php/java/.net/ruby/etc ryan
Re: new to solr
the example.xsl is an example using XSLT to format results. Check: http://wiki.apache.org/solr/XsltResponseWriter For php, check: http://wiki.apache.org/solr/SolPHP ryan Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Thanks so much for your reply! Please tell me what example.xsl is for in conf/xslt. Please let me know where the search result is located. I can use php or .net to display the result in web. Is it created on fly? Thanks, Xiaohui -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Monday, January 14, 2008 11:37 AM To: solr-user@lucene.apache.org Subject: Re: new to solr Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Hello, I am new to solr. Welcome! I followed solr online tutorial to get the example work. The search result is xml. I wonder if there is a way to show result in a form. I saw there is example.xsl in conf/xslt directory. I really don't know how to do it. Anyone has some ideas for me. I really appreciate it! Are you asking how to display results for people to see? A nicely formatted website? Solr (a database) does not aim to solve the display side... but there are lots of clients to help integrate with your website. php/java/.net/ruby/etc ryan
RE: new to solr
Thanks very much, Ryan. I really appreciate it. I will take a look on both. Best regards, Xiaohui -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Monday, January 14, 2008 11:56 AM To: solr-user@lucene.apache.org Subject: Re: new to solr the example.xsl is an example using XSLT to format results. Check: http://wiki.apache.org/solr/XsltResponseWriter For php, check: http://wiki.apache.org/solr/SolPHP ryan Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Thanks so much for your reply! Please tell me what example.xsl is for in conf/xslt. Please let me know where the search result is located. I can use php or .net to display the result in web. Is it created on fly? Thanks, Xiaohui -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: Monday, January 14, 2008 11:37 AM To: solr-user@lucene.apache.org Subject: Re: new to solr Ma, Xiaohui (NIH/NLM/LHC) [C] wrote: Hello, I am new to solr. Welcome! I followed solr online tutorial to get the example work. The search result is xml. I wonder if there is a way to show result in a form. I saw there is example.xsl in conf/xslt directory. I really don't know how to do it. Anyone has some ideas for me. I really appreciate it! Are you asking how to display results for people to see? A nicely formatted website? Solr (a database) does not aim to solve the display side... but there are lots of clients to help integrate with your website. php/java/.net/ruby/etc ryan
Re: new to solr
On Jan 14, 2008 11:55 AM, Ryan McKinley [EMAIL PROTECTED] wrote: the example.xsl is an example using XSLT to format results. Check: http://wiki.apache.org/solr/XsltResponseWriter To add to the above: I think the XsltResponseWriter is not intended for formatting results for display on your web site. Normally you would use your server-side language (PHP, Python, etc.) to query the Solr server, get the results, and format them for display. Solr doesn't provide the front-end search interface for your web site -- you have to create that yourself. -Stuart altlaw.org