Re: solr-map-reduce:jar:6.6.0

[Hrishikesh Gadre]

>>If we do need that functionality somehow, what is the recommended
approach?  Was it replaced by something?

AFAIK it was not replaced by anything on the Solr side. Kite sdk provides
identical functionality and it looks like it supports Solr 6.6. You may
want to try that out

https://github.com/kite-sdk/kite

Thanks
Hrishikesh


On Tue, Aug 29, 2017 at 3:28 PM, John Bickerstaff <j...@johnbickerstaff.com>
wrote:

> If we do need that functionality somehow, what is the recommended
> approach?  Was it replaced by something?
>
> On Tue, Aug 29, 2017 at 4:27 PM, John Bickerstaff <
> j...@johnbickerstaff.com>
> wrote:
>
> > A quick search of the code's imports suggests we aren't actually using
> it.
> > However I see it explicitly mentioned in the POM.
> >
> > I'll have to do more digging to be sure...
> >
> > On Tue, Aug 29, 2017 at 3:44 PM, Hrishikesh Gadre <gadre.s...@gmail.com>
> > wrote:
> >
> >> Hi John,
> >>
> >> The map-reduce contrib module is removed as part of SOLR-9221
> >> <https://issues.apache.org/jira/browse/SOLR-9221>. Are you using this
> >> contrib module?
> >>
> >> Thanks
> >> Hrishikesh
> >>
> >>
> >> On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff <
> >> j...@johnbickerstaff.com>
> >> wrote:
> >>
> >> > All,
> >> >
> >> > I'm looking at upgrading some java projects to solr and lucene 6.6.0.
> >> >
> >> > I get an error when trying to "mvn package" one of them - it
> complained
> >> > about the solr-map-reduce:jar:6.6.0 not being available.
> >> >
> >> > I went here:
> >> > https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
> >> >
> >> > And it looks like this hasn't been updated to 6.6...
> >> >
> >> > Is it going to be updated soon?  Am I misunderstanding something?
> >> >
> >>
> >
> >
>

Re: solr-map-reduce:jar:6.6.0

[Hrishikesh Gadre]

Hi John,

The map-reduce contrib module is removed as part of SOLR-9221
. Are you using this
contrib module?

Thanks
Hrishikesh


On Tue, Aug 29, 2017 at 2:39 PM, John Bickerstaff 
wrote:

> All,
>
> I'm looking at upgrading some java projects to solr and lucene 6.6.0.
>
> I get an error when trying to "mvn package" one of them - it complained
> about the solr-map-reduce:jar:6.6.0 not being available.
>
> I went here:
> https://mvnrepository.com/artifact/org.apache.solr/solr-map-reduce
>
> And it looks like this hasn't been updated to 6.6...
>
> Is it going to be updated soon?  Am I misunderstanding something?
>

Re: Solr config upgrade tool

[Hrishikesh Gadre]

Interesting. This is certainly useful. I was also thinking about having a
tool which can setup a Solr server of intended version and test out the
upgraded configuration.

The validity of configuration files can be checked by creating a Solr
collection (or core) and verifying that it is loaded successfully.

Regarding indexing and querying - I think we can simplify this by having
user provide a sample data for indexing purpose along with set of queries
(along with the expected results) to try out. The tool then can
automatically index the documents, run queries and verify results.

-Hrishikesh

On Fri, Aug 11, 2017 at 1:29 PM, Davis, Daniel (NIH/NLM) [C] <
daniel.da...@nih.gov> wrote:

> Hrishikesh Gadre,
>
> I'm interested in how that might integrate with continuous integration.  I
> briefly worked on a tool to try a configuration out with SolrCloud, e.g.
> upload the config, create a collection, run some stuff, test some stuff.  I
> got the first two working, but not the "run some stuff" and "test some
> stuff" parts of this.
>
> It seemed to me that since the main purposes of config are handlers and
> fields, that using SimplePostTool to import data and such would be useful.
>  I wanted to stick to Java to really validate that this stuff is really as
> easy to do in Java in Python/Node/Ruby.   However, it's not, which means a
> real solution would be opinionated and pick a language.
>
> If curious, you can look at my work at https://github.com/danizen/
> solr-config-tool.   The main innovation is to actually use Junit 4 POJO
> files as tests, because TestController creates a configured test case
> containing all of the test cases.   The other benefit of making these test
> cases in a Maven project, is that a maven based project that is using the
> solr-config-tool command-line could include these tests explicitly in a
> test plan, and their own tests as well.   In this case, they'd parameterize
> their test using properties, and then the SolrConfig object would tell them
> where to find the Solr collection under test.
>
> My intention was for this to help with big-iron investments in Solr, where
> they don't want to spin-up 5 new nodes to test one small schema, which
> would have been the case for my institution.
>
> Maybe I'll give this tool some love soon...
>
> -Original Message-
> From: Hrishikesh Gadre [mailto:gadre.s...@gmail.com]
> Sent: Friday, August 11, 2017 4:17 PM
> To: solr-user@lucene.apache.org; d...@lucene.apache.org
> Subject: Solr config upgrade tool
>
> Hi,
>
> I am currently working on a tool to identify (and in some cases fix) the
> incompatibilities in the Solr configuration (e.g. schema.xml,
> solrconfig.xml etc.) between major versions of Solr.
>
> My goal is to simplify the Solr upgrade process by providing upgrade
> instructions tailored to your configuration. These instuctions can help you
> to answer following questions
>
> - Does my Solr configuration have any backwards incompatible sections? If
> yes which ones?
> - For each of the incompatibility - what do I need to do to fix this
> incompatibility? Where can I get more information about why this
> incompatibility was introduced (e.g. references to Lucene/Solr jira)?
> - Are there any changes in Lucene/Solr which would require me to do a full
> reindexing OR can I get away with an index upgrade?
>
> The initial prototype of this tool is available @
> https://github.com/hgadre/solr-upgrade-tool
>
> For comments (or suggestions) please refer to
> https://issues.apache.org/jira/browse/SOLR-11229
>
> I have implemented upgrade rules for Solr 4 -> Solr 5 and Solr 5 -> Solr 6
> by referring to upgrade instructions in the Solr release notes. But during
> testing I found that many of the configuration changes (e.g. plugin
> deprecations) are not listed in the upgrade section. I am working on
> identifying such configuration changes and adding upgrade rules
> accordingly. I would love any community participation in improving this
> tool.
>
> I will be presenting a talk titled "Apache Solr - Upgrading your upgrade
> experience" @ Lucene revolution this year. Please do attend to know more
> about simplifying Solr upgrade process.
>
> Regards
>
> Hrishikesh
>

Solr config upgrade tool

[Hrishikesh Gadre]

Hi,

I am currently working on a tool to identify (and in some cases fix) the
incompatibilities in the Solr configuration (e.g. schema.xml,
solrconfig.xml etc.) between major versions of Solr.

My goal is to simplify the Solr upgrade process by providing upgrade
instructions tailored to your configuration. These instuctions can help you
to answer following questions

- Does my Solr configuration have any backwards incompatible sections? If
yes which ones?
- For each of the incompatibility - what do I need to do to fix this
incompatibility? Where can I get more information about why this
incompatibility was introduced (e.g. references to Lucene/Solr jira)?
- Are there any changes in Lucene/Solr which would require me to do a full
reindexing OR can I get away with an index upgrade?

The initial prototype of this tool is available @
https://github.com/hgadre/solr-upgrade-tool

For comments (or suggestions) please refer to
https://issues.apache.org/jira/browse/SOLR-11229

I have implemented upgrade rules for Solr 4 -> Solr 5 and Solr 5 -> Solr 6
by referring to upgrade instructions in the Solr release notes. But during
testing I found that many of the configuration changes (e.g. plugin
deprecations) are not listed in the upgrade section. I am working on
identifying such configuration changes and adding upgrade rules
accordingly. I would love any community participation in improving this
tool.

I will be presenting a talk titled "Apache Solr - Upgrading your upgrade
experience" @ Lucene revolution this year. Please do attend to know more
about simplifying Solr upgrade process.

Regards

Hrishikesh

Re: Limiting the number of queries/updates to Solr

[Hrishikesh Gadre]

At one point I was working on SOLR-7344
 (but it fell off the
radar due to various reasons). Specifically I built a servlet request
filter which implements a customizable queuing mechanism using asynchronous
servlet API (Servlet 3 spec). This way you can define how many concurrent
requests of a specific type (e.g. query, indexing etc.) you want to
process. This can also be extended at a core (or collection) level.

https://github.com/hgadre/servletrequest-scheduler


If this is something interesting and useful for the community, I would be
more than happy to help moving this forward. Otherwise I would like to get
any feedback for possible improvements (or drawbacks) etc.

Thanks
Hrishikesh




On Wed, Aug 2, 2017 at 9:45 PM, Walter Underwood 
wrote:

>
> > On Aug 2, 2017, at 8:33 PM, Shawn Heisey  wrote:
> >
> > IMHO, intentionally causing connections to fail when a limit is exceeded
> > would not be a very good idea.  When the rate gets too high, the first
> > thing that happens is all the requests slow down.  The slowdown could be
> > dramatic.  As the rate continues to increase, some of the requests
> > probably would begin to fail.
>
> No, this is a very good idea. It is called “load shedding” or “fail fast”.
> Gracefully dealing with overload is an essential part of system design.
>
> At Netflix, with a pre-Jetty Solr (war file running under Tomcat), we took
> down 40 front end servers with slow response times from the Solr server
> farm. We tied up all the front end threads waiting on responses from the
> Solr servers. That left no front end threads available to respond to
> incoming HTTP requests. It was not a fun evening.
>
> To fix this, we configured the Citrix load balancer to overflow to a
> different server when the outstanding back-end requests hit a limit. The
> overflow server was a virtual server that immediately returned a 503. That
> would free up front end connections and threads in an overload condition.
> The users would get a “search unavailable” page, but the rest of the site
> would continue to work.
>
> Unfortunately, the AWS load balancers don’t offer anything like this, ten
> years later.
>
> The worst case version of this is a stable congested state. It is pretty
> easy to put requests into a queue (connection/server) that are guaranteed
> to time out before they are serviced. If you have 35 requests in the queue,
> a 1 second service time, and a 30 second timeout, those requests are
> already dead when you put them on the queue.
>
> I learned about this when I worked with John Nagle at Ford Aerospace. I
> recommend his note “On Packet Switches with Infinite Storage” (1985) for
> the full story. It is only eight pages long, but packed with goodness.
>
> https://tools.ietf.org/html/rfc970
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
>

Re: SOLR as nosql database store

[Hrishikesh Gadre]

Hi Bharath,

In general its not a good idea to use Solr as the *primary data store* for
various reasons listed here,

https://wiki.apache.org/solr/HowToReindex


But if you design your system such that at-least one copy of the raw data
is stored in some other storage system then you can use Solr as the
operational database.

Hope this helps.

-Hrishikesh




On Mon, May 8, 2017 at 7:43 PM, bharath.mvkumar 
wrote:

> Hi All,
>
> We have a use case where we have mysql database which stores documents and
> also some of the fields in the document is also indexed in solr.
> We plan to move all those documents to solr by making solr as the nosql
> datastore for storing those documents. The reason we plan to do this is
> because we have to support cross center data replication for both mysql and
> solr and we are in a way duplicating the same data.The number of writes we
> do per second is around 10,000. Also currently we have only one shard and
> we
> have around 70 million records and we plan to support close to 1 billion
> records and also perform sharding.
>
> Using solr as the nosql database is a good choice or should we look at
> Cassandra for our use case?
>
> Thanks,
> Bharath Kumar
>
>
>
> --
> View this message in context: http://lucene.472066.n3.
> nabble.com/SOLR-as-nosql-database-store-tp4334119.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: Solr6.3.0 SolrJ API for Basic Authentication

[Hrishikesh Gadre]

Hey,

The other alternative would be to implement a HttpClientConfigurer which
can perform preemptive basic authentication (just the same way SolrRequest
is sending the credentials). The code is available in the branch_6x here,

https://github.com/apache/lucene-solr/blob/1bfa057d5c9d89b116031baa7493ee422b4cbabb/solr/solrj/src/java/org/apache/solr/client/solrj/impl/PreemptiveBasicAuthConfigurer.java

For the 6.3.x installation, you can copy this code and include it as a
custom jar on the client side. As part of the client initialization, just
configure this implementation of client configurer. Here is a sample code
snippet you can use as a reference.

https://github.com/apache/lucene-solr/blob/a986368fd0670840177a8c19fb15dcd1f0e69797/solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java#L111-L123

Let me know if you have any questions.

Thanks
Hrishikesh


On Thu, Feb 16, 2017 at 6:39 AM, Bryan Bende  wrote:

> Hello,
>
> The QueryRequest was just an example, it will work with any request
> that extends SolrRequest.
>
> How are you indexing your documents?
>
> I am going to assume you are doing something like this:
>
> SolrClient client = ...
> client.add(solrInputDocument);
>
> Behind the scenes this will do something like the following:
>
> UpdateRequest req = new UpdateRequest();
> req.add(doc);
> req.setCommitWithin(commitWithinMs);
> req.process(client, collection);
>
> So you can do that your self and first set the basic auth credentials
> on the UpdateRequest which extends SolrRequest.
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 16, 2017 at 5:45 AM, vrindavda  wrote:
> > Hi Bryan,
> >
> > Thanks for your quick response.
> >
> > I am trying to ingest data into SolrCloud,  Hence I will not have any
> solr
> > query. Will it be right approach to use QueryRequest to index data ? Do I
> > need to put any dummy solrQuery instead ?
> >
> >
> >
> > --
> > View this message in context: http://lucene.472066.n3.
> nabble.com/Solr6-3-0-SolrJ-API-for-Basic-Authentication-
> tp4320238p4320675.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>

Re: creating collection using collection API with SSL enabled SolrCloud

[Hrishikesh Gadre]

Hi Sean,

Have you configured the "urlScheme" cluster property (i.e. urlScheme=https)
?
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-CLUSTERPROP:ClusterProperties

Thanks
Hrishikesh



On Thu, Feb 9, 2017 at 8:23 AM, Xie, Sean  wrote:

> Hi All,
>
> When trying to create the collection using the API when there are a few
> replicas, I’m getting error because the call seems to trying to use HTTP
> for the replicas.
>
> https://IP_1:8983/solr/admin/collections?action=CREATE;
> name=My_COLLECTION=1=1&
> collection.configName=my_collection_conf
>
> Here is the error:
>
> org.apache.solr.client.solrj.SolrServerException:IOException occured when
> talking to server at: http://IP_2:8983/solr
>
>
> Is there something need to be configured for that?
>
> Thanks
> Sean
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>

Re: Solrcloud backup delete

[Hrishikesh Gadre]

Hi Johan,

Once the backup is created successfully, Solr does not play any role in
managing the backup copies and it is left up to the user. You may want to
build a script which maintains last N backup copies (and delete old ones).

If you end up building such script, see if you can submit a patch against
SOLR-9744 

Thanks
Hrishikesh


On Wed, Jan 25, 2017 at 11:33 PM, Johan Kooijman 
wrote:

> Hi all,
>
> I see I can easily create/restore backups of the entire collection
> https://cwiki.apache.org/confluence/display/solr/Collections+API.
>
> I now have a situation where these backups fill up a disk, so I need to get
> rid of some. On the URL above I don't see an API call to delete a backup.
>
> What whould be the preferred method for deleting these old backups?
>
> --
> Met vriendelijke groeten / With kind regards,
> Johan Kooijman
>

Re: Can't create collection

[Hrishikesh Gadre]

Based on the stack trace, it looks like the Solr collection create request
was not processed (or in-progress) on the overseer node. Can you check if
the Solr server hosting the overseer role is healthy and is connected to
Zookeeper ?

If there are no issues with the cluster, you can try using the async mode
for this operation.
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-AsynchronousCalls


Thanks
Hrishikesh


On Mon, Jan 16, 2017 at 11:36 AM, Joe Obernberger <
joseph.obernber...@gmail.com> wrote:

> Hi All - trying to create a collection in Solr 6.3.0 on a 5 node cluster
> and getting a timeout error.  The cluster has a collection already, but I'm
> trying to make a new one.
>
> The UI reports 'Connection to Solr lost - Please check the Solr
> instance.'.  It is still running, but the log reports:
>
> 2017-01-16 19:34:07.865 INFO  (qtp606548741-235) [   ]
> o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params
> replicationFactor=1=compositeId
> configName=MINTEST=1=Space
> =compositeId=CREATE=1=json&_=1484595286393 and
> sendToOCPQueue=true
> 2017-01-16 19:34:14.485 ERROR (qtp606548741-208) [   ]
> o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create
> the collection time out:180s
> at org.apache.solr.handler.admin.CollectionsHandler.handleRespo
> nse(CollectionsHandler.java:292)
> at org.apache.solr.handler.admin.CollectionsHandler.handleReque
> stBody(CollectionsHandler.java:218)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(Req
> uestHandlerBase.java:153)
> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(Http
> SolrCall.java:660)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:
> 441)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDisp
> atchFilter.java:303)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDisp
> atchFilter.java:254)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilte
> r(ServletHandler.java:1668)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHan
> dler.java:581)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped
> Handler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHa
> ndler.java:548)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(
> SessionHandler.java:226)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(
> ContextHandler.java:1160)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHand
> ler.java:511)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(
> SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(
> ContextHandler.java:1092)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(Scoped
> Handler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.ha
> ndle(ContextHandlerCollection.java:213)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(
> HandlerCollection.java:119)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(Handl
> erWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.
> java:308)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConne
> ction.java:244)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.
> succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.
> java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChann
> elEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume
> .produceAndRun(ExecuteProduceConsume.java:246)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume
> .run(ExecuteProduceConsume.java:156)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(Queued
> ThreadPool.java:654)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedT
> hreadPool.java:572)
> at java.lang.Thread.run(Thread.java:745)
>
> Thanks for any ideas!
>
> -Joe
>

Re: Rolling backups of a collection

[Hrishikesh Gadre]

Hi Mike,

I filed SOLR-9744 <https://issues.apache.org/jira/browse/SOLR-9744> to
track this work. Please comment on this jira if you have any suggestions.

Thanks
Hrishikesh


On Wed, Nov 9, 2016 at 11:07 AM, Hrishikesh Gadre <gadre.s...@gmail.com>
wrote:

> Hi Mike,
>
> Currently we don't have capability to take rolling backups for the Solr
> collections. I think it should be fairly straightforward to write a script
> that implements this functionality outside of Solr. If you post that
> script, may be we can even ship it as part of Solr itself (for the benefit
> of the community).
>
> Thanks
> Hrishikesh
>
>
>
> On Wed, Nov 9, 2016 at 9:17 AM, Mike Thomsen <mikerthom...@gmail.com>
> wrote:
>
>> I read over the docs (
>> https://cwiki.apache.org/confluence/display/solr/Making+and+
>> Restoring+Backups)
>> and am not quite sure what route to take. My team is looking for a way to
>> backup the entire index of a SolrCloud collection with regular rotation
>> similar to the backup option available in a single node deployment.
>>
>> We have plenty of space in our HDFS cluster. Resources are not an issue in
>> the least to have a rolling back up of say, the last seven days. Is there
>> a
>> good way to implement this sort of rolling backup with the APIs or will we
>> have to roll some of the functionality ourselves?
>>
>> I'm not averse to using the API to dump a copy of each shard to HDFS.
>> Something like this:
>>
>> /solr/collection/replication?command=backup=shard_1_1=7
>>
>> Is that a viable route to achieve this or do we need to do something else?
>>
>> Thanks,
>>
>> Mike
>>
>
>

Re: Rolling backups of a collection

[Hrishikesh Gadre]

Hi Mike,

Currently we don't have capability to take rolling backups for the Solr
collections. I think it should be fairly straightforward to write a script
that implements this functionality outside of Solr. If you post that
script, may be we can even ship it as part of Solr itself (for the benefit
of the community).

Thanks
Hrishikesh



On Wed, Nov 9, 2016 at 9:17 AM, Mike Thomsen  wrote:

> I read over the docs (
> https://cwiki.apache.org/confluence/display/solr/
> Making+and+Restoring+Backups)
> and am not quite sure what route to take. My team is looking for a way to
> backup the entire index of a SolrCloud collection with regular rotation
> similar to the backup option available in a single node deployment.
>
> We have plenty of space in our HDFS cluster. Resources are not an issue in
> the least to have a rolling back up of say, the last seven days. Is there a
> good way to implement this sort of rolling backup with the APIs or will we
> have to roll some of the functionality ourselves?
>
> I'm not averse to using the API to dump a copy of each shard to HDFS.
> Something like this:
>
> /solr/collection/replication?command=backup=shard_1_1=7
>
> Is that a viable route to achieve this or do we need to do something else?
>
> Thanks,
>
> Mike
>

Re: Challenges with new Solrcloud Backup/Restore functionality

[Hrishikesh Gadre]

Hi Stephen,

Thanks for the update.

Regarding SOLR-9527 - I think we need a unit test for verifying
"createNodeSet" functionality. I will spend some time on it in next couple
of days.

Also regarding #2, i also found similar issue (doc count mismatch after
restore) while testing with a large collection (~50GB index size). I have
opened SOLR-9598 to track this. Please take a look and comment if you have
any insight.

-Hrishikesh

On Tue, Nov 8, 2016 at 12:54 PM, Stephen Weiss <steve.we...@wgsn.com> wrote:

> Just wanted to note that we tested out the patch from SOLR-9527 and it
> worked perfectly for the balancing issue - thank you so much for that!
>
> As for issue #2, we've resorted to doing a hard commit, stopping all
> indexing against the index, and then taking the backup, and we have a
> reasonably good success rate with that.  The system is set up to
> automatically delete and retry the backup/restore process if the cores
> don't match, so that's allowed us to smooth over that problem and get this
> process out into production.   We've been using it for several weeks now
> without any major issue!
>
> We just looked because Solr 6.3 was out, and wanted to know if we could
> upgrade without patching again, but it appears this ticket hasn't gone
> anywhere yet.  I know one users' testing is probably not enough, but given
> that it seems the patch works just fine, are there any plans to merge it
> into release yet?
>
> --
> Steve
>
> On Tue, Oct 4, 2016 at 6:46 PM, Stephen Lewis <sle...@panopto.com sle...@panopto.com>> wrote:
> Hi All,
>
> I have been experiencing error#1 too with the current branch_6_2 build. I
> started noticing after I applied my patch to that branch<
> https://issues.apache.org/jira/browse/SOLR-9527> (on issue #2), but it
> appears to occur without the patch as well. I haven't seen this issue with
> solr 6.1.0 despite extensive testing. I haven't confirmed if this occurs on
> the official 6.2.0 release build. I will try to confirm and gather more
> data soon.
>
> As with Stephen Weiss, I also am not seeing any errors logged in the index
> after backup and the task is marked as succeeded. However, after each
> backup which is missing a large amount of data, the restore command fails,
> in the sense that the collection is created, but the initialized cores are
> blank and the logs contain errors about "incomplete segments". I will try
> to research further and get back with more data soon.
>
>
>
> On Mon, Sep 26, 2016 at 11:26 AM, Hrishikesh Gadre <gadre.s...@gmail.com
> <mailto:gadre.s...@gmail.com>> wrote:
> Hi Stephen,
>
> regarding #1, can you verify following steps during backup/restore?
>
> - Before backup command, make sure to run a "hard" commit on the original
> collection. The backup operation will capture only hard committed data.
> - After restore command, check the Solr web UI to verify that all replicas
> of the new (or restored) collection are in the "active" state. During my
> testing, I found that when one or more replicas are in "recovery" state,
> the doc count of the restored collection doesn't match the doc count of the
> original collection. But after the recovery is complete, the doc counts
> match. I will file a JIRA to fix this issue.
>
> Thanks
> Hrishikesh
>
> On Mon, Sep 26, 2016 at 9:34 AM, Stephen Weiss <steve.we...@wgsn.com
> <mailto:steve.we...@wgsn.com>> wrote:
>
> > #2 - that's great news.  I'll try to patch it in and test it out.
> >
> > #1 - In all cases, the backup and restore both appear successful.  There
> > are no failure messages for any of the shards, no warnings, etc - I
> didn't
> > even realize at first that data was missing until I noticed differences
> in
> > some of the query results when we were testing.  Either manual restore of
> > the data or using the restore API (with all data on one node), we see the
> > same, so I think it's more a problem in the backup process than the
> restore
> > process.
> >
> > If there's any kind of debugging output we can provide that can help
> solve
> > this, let me know.
> >
> > --
> > Steve
> >
> > On Sun, Sep 25, 2016 at 7:17 PM, Hrishikesh Gadre <gadre.s...@gmail.com
> <mailto:gadre.s...@gmail.com>>
> > wrote:
> >
> >> Hi Steve,
> >>
> >> Regarding the 2nd issue, a JIRA is already created and patch is uploaded
> >> (SOLR-9527). Can someone review and commit the patch?
> >>
> >> Regarding 1st issue, does backup command succeeds? Also do you see any
> >> warning/error log messages? How about the restore command?
> >&g

Re: Backup to HDFS while running cluster on local disk

[Hrishikesh Gadre]

Hi Mike,

Thanks for bringing this up. You can certainly backup the index data stored
on local file-system to HDFS.

The HDFS backup repository implementation uses the same configuration
properties as expected by the HDFS directory factory. Here is the
description of the parameters,

   - location (Optional) - This configuration parameter defines the default
   location where the backups can be stored. If this parameter is not
   configured, then you will need to explicitly specify the location parameter
   to your backup and restore commands.
   - solr.hdfs.home (Required) - This configuration parameter defines the
   fully qualified URI for the root path of HDFS. e.g. hdfs://name-node-1/. In
   case the index files are also stored on HDFS, this path refers to the
   directory used to store index files in HDFS e.g. hdfs://name-node-1/solr
   - solr.hdfs.confdir (Optional) - A directory (on local file-system)
   which contains the configuration files for HDFS (e.g. hdfs-site.xml,
   core-site.xml etc.)


I will also update the docs accordingly.

-Hrishikesh


On Tue, Nov 8, 2016 at 3:36 AM, Mike Thomsen  wrote:

> We have SolrCloud running on bare metal but want the nightly snapshots to
> be written to HDFS. Can someone give me some help on configuring the
> HdfsBackupRepository?
>
> 
>  "org.apache.solr.core.backup.repository.HdfsBackupRepository" default=
> "false">
> ${solr.hdfs.default.backup.path}
> ${solr.hdfs.home:}
> ${solr.hdfs.confdir:}
> 
> 
>
> Not sure how to proceed on configuring this because the documentation is a
> bit sparse on what some of those values mean in this context. The example
> looked geared toward someone using HDFS both to store the index and do
> backup/restore.
>
> Thanks,
>
> Mike
>

Re: Public/Private data in Solr :: Metadata or ?

[Hrishikesh Gadre]

As part of Cloudera Search, we have integrated with Apache Sentry for
document level authorization. Currently we are using custom search
component to implement filtering. Please refer to this blog post for
details,
http://blog.cloudera.com/blog/2014/07/new-in-cdh-5-1-document-level-security-for-cloudera-search/

I am currently working on a Sentry based plugin implementation which can be
hooked in the Solr authorization framework. Currently Solr authorization
framework doesn't implement document level security. I filed SOLR-9578
 to add the relevant doc
level security support in Solr.

The main drawback of custom search component based mechanism is that it
requires a special solrconfig.xml file (which is using these custom search
components). On the other hand, once Solr provides hooks to implement doc
level security as part of authorization framework, then this restriction
will go away.

If you have any ideas (or concerns) with this feature, please feel free to
comment on the jira.

Thanks
Hrishikesh

On Wed, Oct 19, 2016 at 7:48 AM, Shawn Heisey  wrote:

> On 10/18/2016 3:00 PM, John Bickerstaff wrote:
> > How (or is it even wise) to "segregate data" in Solr so that some data
> > can be seen by some users and some data not be seen?
>
> IMHO, security like this isn't really Solr's job ... but with the right
> data in the index, the system that DOES handle the security can include
> a filter with each user's query to restrict them to only the data they
> are allowed to see.  There are many ways to put data in the index for
> efficient use by a filter.  The simplest would be a boolean field with a
> name like isPublic or isPrivate, where true and false are mapped as
> necessary to public and private.
>
> Naturally, the users must not be able to reach Solr directly ... they
> must be restricted to the software that connects to Solr on their
> behalf.  Blocking end users from direct network access to Solr is a good
> idea even if there are no other security needs.
>
> There are more comprehensive solutions available, as you will notice
> from other replies, but the idea of simple filtering, controlled by your
> application, should work.
>
> Thanks,
> Shawn
>
>

Re: Challenges with new Solrcloud Backup/Restore functionality

[Hrishikesh Gadre]

Hi Stephen,

regarding #1, can you verify following steps during backup/restore?

- Before backup command, make sure to run a "hard" commit on the original
collection. The backup operation will capture only hard committed data.
- After restore command, check the Solr web UI to verify that all replicas
of the new (or restored) collection are in the "active" state. During my
testing, I found that when one or more replicas are in "recovery" state,
the doc count of the restored collection doesn't match the doc count of the
original collection. But after the recovery is complete, the doc counts
match. I will file a JIRA to fix this issue.

Thanks
Hrishikesh

On Mon, Sep 26, 2016 at 9:34 AM, Stephen Weiss <steve.we...@wgsn.com> wrote:

> #2 - that's great news.  I'll try to patch it in and test it out.
>
> #1 - In all cases, the backup and restore both appear successful.  There
> are no failure messages for any of the shards, no warnings, etc - I didn't
> even realize at first that data was missing until I noticed differences in
> some of the query results when we were testing.  Either manual restore of
> the data or using the restore API (with all data on one node), we see the
> same, so I think it's more a problem in the backup process than the restore
> process.
>
> If there's any kind of debugging output we can provide that can help solve
> this, let me know.
>
> --
> Steve
>
> On Sun, Sep 25, 2016 at 7:17 PM, Hrishikesh Gadre <gadre.s...@gmail.com>
> wrote:
>
>> Hi Steve,
>>
>> Regarding the 2nd issue, a JIRA is already created and patch is uploaded
>> (SOLR-9527). Can someone review and commit the patch?
>>
>> Regarding 1st issue, does backup command succeeds? Also do you see any
>> warning/error log messages? How about the restore command?
>>
>> Thanks
>> Hrishikesh
>>
>>
>>
>> On Sat, Sep 24, 2016 at 12:14 PM, Stephen Weiss <steve.we...@wgsn.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> We're very excited about SolrCloud's new backup / restore collection
>>> APIs, which should introduce some major new efficiencies into our indexing
>>> workflow.  Unfortunately, we've run into some snags with it that are
>>> preventing us from moving into production.  I was hoping someone on the
>>> list could help.
>>>
>>> 1) Data inconsistencies
>>>
>>> There seems to be a problem getting all the data consistently.
>>> Sometimes, the backup will contain all of the data in the collection, and
>>> sometimes, large portions of the collection (as much as 40%) will be
>>> missing.  We haven't quite figured out what might cause this yet, although
>>> one thing I've noticed is the chances of success are greater when we are
>>> only backing up one collection at a time.  Unfortunately, for our workflow,
>>> it will be difficult to make that work, and there still doesn't seem to be
>>> a guarantee of success either way.
>>>
>>> 2) Shards are not distributed
>>>
>>> To make matters worse, for some reason, any type of restore operation
>>> always seems to put all shards of the collection on the same node.  We've
>>> tried setting maxShardsPerNode to 1 in the restore command, but this has no
>>> effect.  We are seeing the same behavior on both 6.1 and 6.2.1.  No matter
>>> what we do, all the shards always go to the same node - and it's not even
>>> the node that we execute the restore request on, but oddly enough, a
>>> totally different node, and always the same one (the 4th one).  It should
>>> be noted that all nodes of our 8 node cloud are up and totally functional
>>> when this happens.
>>>
>>> To work around this, we wrote up a quick script to create an empty
>>> collection, which always distributes itself across the cloud quite well
>>> (another indication that there's nothing wrong with the nodes themselves),
>>> and then we rsync the individual shards' data into the empty shards and
>>> reload the collection.  This works fine, however, because of the data
>>> inconsistencies mentioned above, we can't really move forward anyway.
>>>
>>>
>>> Problem #2, we have a reasonable workaround for, but problem #1 we do
>>> not.  If anyone has any thoughts about either of these problems, I would be
>>> very grateful.  Thanks!
>>>
>>> --
>>> Steve
>>>
>>> 
>>>
>>> WGSN is a global foresight business. Our experts provide deep insight
>>> and analysis of co

Re: Challenges with new Solrcloud Backup/Restore functionality

[Hrishikesh Gadre]

Hi Steve,

Regarding the 2nd issue, a JIRA is already created and patch is uploaded
(SOLR-9527). Can someone review and commit the patch?

Regarding 1st issue, does backup command succeeds? Also do you see any
warning/error log messages? How about the restore command?

Thanks
Hrishikesh



On Sat, Sep 24, 2016 at 12:14 PM, Stephen Weiss 
wrote:

> Hi everyone,
>
> We're very excited about SolrCloud's new backup / restore collection APIs,
> which should introduce some major new efficiencies into our indexing
> workflow.  Unfortunately, we've run into some snags with it that are
> preventing us from moving into production.  I was hoping someone on the
> list could help.
>
> 1) Data inconsistencies
>
> There seems to be a problem getting all the data consistently.  Sometimes,
> the backup will contain all of the data in the collection, and sometimes,
> large portions of the collection (as much as 40%) will be missing.  We
> haven't quite figured out what might cause this yet, although one thing
> I've noticed is the chances of success are greater when we are only backing
> up one collection at a time.  Unfortunately, for our workflow, it will be
> difficult to make that work, and there still doesn't seem to be a guarantee
> of success either way.
>
> 2) Shards are not distributed
>
> To make matters worse, for some reason, any type of restore operation
> always seems to put all shards of the collection on the same node.  We've
> tried setting maxShardsPerNode to 1 in the restore command, but this has no
> effect.  We are seeing the same behavior on both 6.1 and 6.2.1.  No matter
> what we do, all the shards always go to the same node - and it's not even
> the node that we execute the restore request on, but oddly enough, a
> totally different node, and always the same one (the 4th one).  It should
> be noted that all nodes of our 8 node cloud are up and totally functional
> when this happens.
>
> To work around this, we wrote up a quick script to create an empty
> collection, which always distributes itself across the cloud quite well
> (another indication that there's nothing wrong with the nodes themselves),
> and then we rsync the individual shards' data into the empty shards and
> reload the collection.  This works fine, however, because of the data
> inconsistencies mentioned above, we can't really move forward anyway.
>
>
> Problem #2, we have a reasonable workaround for, but problem #1 we do
> not.  If anyone has any thoughts about either of these problems, I would be
> very grateful.  Thanks!
>
> --
> Steve
>
> 
>
> WGSN is a global foresight business. Our experts provide deep insight and
> analysis of consumer, fashion and design trends. We inspire our clients to
> plan and trade their range with unparalleled confidence and accuracy.
> Together, we Create Tomorrow.
>
> WGSN is part of WGSN Limited, comprising of
> market-leading products including WGSN.com, WGSN
> Lifestyle & Interiors, WGSN
> INstock, WGSN StyleTrial com/en/styletrial/> and WGSN Mindset en/services/consultancy/>, our bespoke consultancy services.
>
> The information in or attached to this email is confidential and may be
> legally privileged. If you are not the intended recipient of this message,
> any use, disclosure, copying, distribution or any action taken in reliance
> on it is prohibited and may be unlawful. If you have received this message
> in error, please notify the sender immediately by return email and delete
> this message and any copies from your computer and network. WGSN does not
> warrant that this email and any attachments are free from viruses and
> accepts no liability for any loss resulting from infected email
> transmissions.
>
> WGSN reserves the right to monitor all email through its networks. Any
> views expressed may be those of the originator and not necessarily of WGSN.
> WGSN is powered by Ascential plc, which
> transforms knowledge businesses to deliver exceptional performance.
>
> Please be advised all phone calls may be recorded for training and quality
> purposes and by accepting and/or making calls from and/or to us you
> acknowledge and agree to calls being recorded.
>
> WGSN Limited, Company number 4858491
>
> registered address:
>
> Ascential plc, The Prow, 1 Wilder Walk, London W1B 5AP
>
> WGSN Inc., tax ID 04-3851246, registered office c/o National Registered
> Agents, Inc., 160 Greentree Drive, Suite 101, Dover DE 19904, United States
>
> 4C Serviços de Informação Ltda., CNPJ/MF (Taxpayer's Register):
> 15.536.968/0001-04, Address: Avenida Cidade Jardim, 377, 7˚ andar CEP
> 01453-000, Itaim Bibi, São Paulo
>
> 4C Business Information Consulting (Shanghai) Co., Ltd, 富新商务信息咨询（上海）有限公司,
> registered address Unit 4810/4811, 48/F Tower 1, Grand Gateway, 1 Hong 

Re: Distributing nodes with the collections API RESTORE command

[Hrishikesh Gadre]

Hi Stephen,

Thanks for the update. I filed SOLR-9527
 for tracking purpose. I
will take a look and get back to you.

Thanks
Hrishikesh

On Fri, Sep 16, 2016 at 2:56 PM, Stephen Lewis  wrote:

> Hello,
>
> I've tried this on both solr 6.1 and 6.2, with the same result. You are
> right that the collections API offering collection level backup/restore
> from remote server is a new feature.
>
> After some more experimentation, I am fairly sure that this is a bug which
> is specific to the leaders in backup restore. After I ran a command to
> restore a backup of the collection "foo" (which has maxShardsPerNode set to
> 1 as well) with a replication factor of 2, I see consistently that the
> followers (replica > 1) are correctly distributed, but all of the leaders
> are brought up hosted on one shard.
>
> *Repro*
>
> *create *
> http://solr.test:8983/solr/admin/collections?action=
> CREATE=foo=3=1
> configName=test-one
> (after creation, all shards are on different nodes as expected)
>
> *backup*
> http://solr.test:8983/solr/admin/collections?action=
> BACKUP=foo-2=foo=foo-2
>
> *delete*
> http://solr.test:8983/solr/admin/collections?action=DELETE=foo
>
> *restore*
> Result: All leaders are hosted on node, followers are spread about.
>
>  {
>   "responseHeader" : { "status" : 0,"QTime" : 7},
>   "cluster" : {
> "collections" : {
>   "foo" : {
> "replicationFactor" : "2",
> "shards" : {
>   "shard2" : {
> "range" : "d555-2aa9",
> "state" : "active",
> "replicas" : {
>   "core_node1" : {
> "core" : "foo_shard2_replica0",
> "base_url" : "http://IP1:8983/solr;,
> "node_name" : "IP1:8983_solr",
> "state" : "active",
> "leader" : "true"
>   },
>   "core_node4" : {
> "core" : "foo_shard2_replica1",
> "base_url" : "http://IP2:8983/solr;,
> "node_name" : "IP2:8983_solr",
> "state" : "recovering"
>   }
> }
>   },
>   "shard3" : {
> "range" : "2aaa-7fff",
> "state" : "active",
> "replicas" : {
>   "core_node2" : {
> "core" : "foo_shard3_replica0",
> "base_url" : "http://IP1:8983/solr;,
> "node_name" : "IP1:8983_solr",
> "state" : "active",
> "leader" : "true"
>   },
>   "core_node5" : {
> "core" : "foo_shard3_replica1",
> "base_url" : "http://IP3:8983/solr;,
> "node_name" : "IP3:8983_solr",
> "state" : "recovering"
>   }
> }
>   },
>   "shard1" : {
> "range" : "8000-d554",
> "state" : "active",
> "replicas" : {
>   "core_node3" : {
> "core" : "foo_shard1_replica0",
> "base_url" : "http://IP1:8983/solr;,
> "node_name" : "IP1:8983_solr",
> "state" : "active",
> "leader" : "true"
>   },
>   "core_node6" : {
> "core" : "foo_shard1_replica1",
> "base_url" : "http://IP4:8983/solr;,
> "node_name" : "IP4:8983_solr",
> "state" : "recovering"
>   }
> }
>   }
> },
> "router" : {
>   "name" : "compositeId"
> },
> "maxShardsPerNode" : "1",
> "autoAddReplicas" : "false",
> "znodeVersion" : 204,
> "configName" : "test-one"
>   }
> },
> "properties" : {
>   "location" : "/mnt/solr_backups"
> },
> "live_nodes" : [
>   "IP5:8983_solr",
>   "IP3:8983_solr",
>   "IP6:8983_solr",
>   "IP4:8983_solr",
>   "IP7:8983_solr",
>   "IP1:8983_solr",
>   "IP8:8983_solr",
>   "IP9:8983_solr",
>   "IP2:8983_solr"]
>   }
> }
>
>
> On Fri, Sep 16, 2016 at 2:07 PM, Reth RM  wrote:
>
> > Which version of solr? Afaik, until 6.1, solr backup and restore command
> > apis required to do separate backup for each shard, and then restore in
> > similar lines( both go for each). 6.1 version seems to have new feature
> of
> > backing up entire collection records and then restoring it back to new
> > collection setup(did not try yet).
> >
> >
> > On Thu, Sep 15, 2016 at 1:45 PM, Stephen Lewis 
> wrote:
> >
> > > Hello,
> > >
> > > I have a solr cloud cluster in a test environment running 6.1 where I
> am
> > > looking at using the collections API BACKUP and RESTORE commands to
> > manage
> > > data integrity.
> > >
> > > When restoring from a backup, I'm finding the same behavior occurs
> every
> > > time; after the restore command, 

Re: Issues with Solr 6.2 Backup/Restore feature

[Hrishikesh Gadre]

Hi Georg,

It looks like "_2ag9.si" file is missing in the snapshot.shard1 folder of
the backup. Can you try running Lucene CheckIndex tool on the
snapshot.shard1 folder ?

Also can you post the logs during the backup creation as well?

Thanks
Hrishikesh






On Tue, Sep 13, 2016 at 7:12 PM, Georg Bottenhofer <
georg.bottenho...@x75.net> wrote:

> Hi.
>
> We have two Systems running in different Datacenters. The second system is
> a failover system with about one hour difference in Data what is OK for us.
>
> Until Solr 5.5.3 we used a hack with the „replication/snapshot“-Tool to
> copy the indizes over and it worked quite well. A few days ago we upgraded
> our staging-system to Solr 6.2.0 and wanted to try the new collections
> backup/restore tool.
>
> First I tried to backup an index with about 50MByte and restored it
> multiple times to another solr-system with success. Now I took a copy of
> the Live-System (about 350MB) and failed.
>
> I run the Backup with:
>
> curl 'http://localhost:8983/solr/admin/collections?action=
> BACKUP=mybackup=collection1=/backup/'
>
> and wanted to restore it the other system after copying the files via tgz
> to the same directory structure with:
>
> curl 'http://localhost:8983/solr/admin/collections?action=
> RESTORE=mybackup=/tmp/backup/=collection1'
>
> I pasted a copy of the Error here:
> http://pastebin.com/T1C2BxcC
>
> I already tried to get help in the IRC-Channel, but elyograg mentioned I
> have to post this at the mailing list, hoping you could give me a clue what
> I am doing wrong.
>
> All systems are running with:
> - Solr 6.2.0
> - Zookeeper 3.4.9
> - java version „1.8.0_101"
> - 16 GB RAM
>
> Thanks for your help in advance.
>
> Mit freundlichem Gruß
> Georg Bottenhofer
>
> --
> Kennst du schon WerStreamt.es?
>
> Prüfe die Verfügbarkeit von Filmen und Serien
> bei iTunes, Watchever, Maxdome u.v.m.
>
> Im Web und als App: https://www.werstreamt.es/
> --
>
>
> x75 GmbH
> Onlinelösungen | Werbekonzeption | Designleistungen
>
> Orleansstraße 45a
> 81667 München
>
> T. 089 6244 759-63
> F. 089 6244 759-860
>
> georg.bottenho...@x75.net
>
>
> --
> x75 GmbH
> Sitz: Orleansstraße 45a, 81667 München Amtsgericht München HRB 178409
> Geschäftsführer: Johannes Hammersen
> Ust-Id: DE264251090
>
>

Re: proper routing (from non-Java client) in solr cloud 5.0.0

[Hrishikesh Gadre]

Hi Ian,

As per my understanding, Solrj does not use Zookeeper watches but instead
caches the information (along with a TTL). You can find more information
here,

https://issues.apache.org/jira/browse/SOLR-5473
https://issues.apache.org/jira/browse/SOLR-5474

Regards
Hrishikesh


On Tue, Apr 14, 2015 at 8:49 AM, Ian Rose ianr...@fullstory.com wrote:

 Hi all -

 I've just upgraded my dev install of Solr (cloud) from 4.10 to 5.0.  Our
 client is written in Go, for which I am not aware of a client, so we wrote
 our own.  One tricky bit for this was the routing logic; if a document has
 routing prefix X and belong to collection Y, we need to know which solr
 node to connect to.  Previously we accomplished this by watching the
 clusterstate.json
 file in zookeeper - at startup and whenever it changes, the client parses
 the file contents to build a routing table.

 However in 5.0 newly create collections do not show up in clusterstate.json
 but instead have their own state.json document.  Are there any
 recommendations for how to handle this from the client?  The obvious answer
 is to watch every collection's state.json document, but we run a lot of
 collections (~1000 currently, and growing) so I'm concerned about keeping
 that many watches open at the same time (should I be?).  How does the SolrJ
 client handle this?

 Thanks!
 - Ian

Re: Collections API - HTTP verbs

[Hrishikesh Gadre]

Thanks Mark and Scott. Adding quotes around the URL fixed the problem.

Regards
Hrishikesh

On Thu, Feb 19, 2015 at 7:30 AM, Scott Dawson sc.e.daw...@gmail.com wrote:

 Hrishikesh,
 If you're running on Linux or Unix, the first ampersand in the URL is
 interpreted as the shell's run this in the background operator and
 anything beyond the ampersand will not be passed to curl. So Mark is right
 -- put single quotes around the URL so that it's not interpreted by the
 shell.

 Regards,
 Scott

 On Wed, Feb 18, 2015 at 9:31 PM, Mark Miller markrmil...@gmail.com
 wrote:

  Perhaps try quotes around the url you are providing to curl. It's not
  complaining about the http method - Solr has historically always taken
  simple GET's for http - for good or bad, you pretty much only post
  documents / updates.
 
  It's saying the name param is required and not being found and since you
  are trying to specify the name, I'm guessing something about the command
 is
  not working. You might try just shoving it in a browser url bar as well.
 
  - Mark
 
  On Wed Feb 18 2015 at 8:56:26 PM Hrishikesh Gadre gadre.s...@gmail.com
  wrote:
 
   Hi,
  
   Can we please document which HTTP method is supposed to be used with
 each
   of these APIs?
  
   https://cwiki.apache.org/confluence/display/solr/Collections+API
  
   I am trying to invoke following API
  
   curl http://
  
 hostname:8983/solr/admin/collections?action=CLUSTERPROPname=urlScheme
   val=https
  
   This request is failing due to following error,
  
   2015-02-18 17:29:39,965 INFO
 org.apache.solr.servlet.SolrDispatchFilter:
   [admin] webapp=null path=/admin/collections params={action=CLUSTERPROP}
   status=400 QTime=20
  
   org.apache.solr.core.SolrCore: org.apache.solr.common.SolrException:
   Missing required parameter: name
  
   at
   org.apache.solr.common.params.RequiredSolrParams.get(
   RequiredSolrParams.java:49)
  
   at
   org.apache.solr.common.params.RequiredSolrParams.check(
   RequiredSolrParams.java:153)
  
   at
   org.apache.solr.handler.admin.CollectionsHandler.handleProp(
   CollectionsHandler.java:238)
  
   at
   org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
   CollectionsHandler.java:200)
  
   at
   org.apache.solr.handler.RequestHandlerBase.handleRequest(
   RequestHandlerBase.java:135)
  
   at
   org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(
   SolrDispatchFilter.java:770)
  
   at
   org.apache.solr.servlet.SolrDispatchFilter.doFilter(
   SolrDispatchFilter.java:271)
  
   I am using Solr 4.10.3 version.
  
   Thanks
  
   Hrishikesh
  
 

Collections API - HTTP verbs

[Hrishikesh Gadre]

Hi,

Can we please document which HTTP method is supposed to be used with each
of these APIs?

https://cwiki.apache.org/confluence/display/solr/Collections+API

I am trying to invoke following API

curl http://
hostname:8983/solr/admin/collections?action=CLUSTERPROPname=urlSchemeval=https

This request is failing due to following error,

2015-02-18 17:29:39,965 INFO org.apache.solr.servlet.SolrDispatchFilter:
[admin] webapp=null path=/admin/collections params={action=CLUSTERPROP}
status=400 QTime=20

org.apache.solr.core.SolrCore: org.apache.solr.common.SolrException:
Missing required parameter: name

at
org.apache.solr.common.params.RequiredSolrParams.get(RequiredSolrParams.java:49)

at
org.apache.solr.common.params.RequiredSolrParams.check(RequiredSolrParams.java:153)

at
org.apache.solr.handler.admin.CollectionsHandler.handleProp(CollectionsHandler.java:238)

at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:200)

at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)

at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:770)

at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:271)

I am using Solr 4.10.3 version.

Thanks

Hrishikesh

Re: Checkout the source Code to the Release Version of Solr?

[Hrishikesh Gadre]

Hi,

You can get the released code base here

https://github.com/apache/lucene-solr/releases

Thanks
Hrishikesh

On Tue, Feb 17, 2015 at 2:20 PM, O. Olson olson_...@yahoo.it wrote:

 At this time the latest released version of Solr is 4.10.3. Is there anyway
 we can get the source code for this release version?

 I tried to checkout the Solr code from
 http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_10/ In
 the
 commit log, I see a number of revisions but nothing mention which is the
 release version. The latest revision being 1657441 on Feb 4. Does this
 correspond to 4.10.3? If no, then how do I go about getting the source code
 of 4.10.3.

 I'm also curious where the version number is embedded i.e. is it in a file
 somewhere?

 I want to ensure I am using the released version, and not some bug fixes
 after the version got released.

 Thank you in anticipation.




 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Checkout-the-source-Code-to-the-Release-Version-of-Solr-tp4187041.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: Checkout the source Code to the Release Version of Solr?

[Hrishikesh Gadre]

Also the version number is encoded (at least) in the build file

https://github.com/apache/lucene-solr/blob/817303840fce547a1557e330e93e5a8ac0618f34/lucene/common-build.xml#L32

Hope this helps.

Thanks
Hrishikesh

On Tue, Feb 17, 2015 at 2:25 PM, Hrishikesh Gadre gadre.s...@gmail.com
wrote:

 Hi,

 You can get the released code base here

 https://github.com/apache/lucene-solr/releases

 Thanks
 Hrishikesh

 On Tue, Feb 17, 2015 at 2:20 PM, O. Olson olson_...@yahoo.it wrote:

 At this time the latest released version of Solr is 4.10.3. Is there
 anyway
 we can get the source code for this release version?

 I tried to checkout the Solr code from
 http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_10/ In
 the
 commit log, I see a number of revisions but nothing mention which is the
 release version. The latest revision being 1657441 on Feb 4. Does this
 correspond to 4.10.3? If no, then how do I go about getting the source
 code
 of 4.10.3.

 I'm also curious where the version number is embedded i.e. is it in a file
 somewhere?

 I want to ensure I am using the released version, and not some bug fixes
 after the version got released.

 Thank you in anticipation.




 --
 View this message in context:
 http://lucene.472066.n3.nabble.com/Checkout-the-source-Code-to-the-Release-Version-of-Solr-tp4187041.html
 Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrCloud - Enable SSL

[Hrishikesh Gadre]

OK. I think I have figured this out.

https://issues.apache.org/jira/browse/SOLR-5610

On Thu, Jan 15, 2015 at 6:00 PM, Hrishikesh Gadre gadre.s...@gmail.com
wrote:

 Hi,

 If we need to enable SSL configuration for an existing Solr cluster
 (hosting one or more collections), do we need to manually update the
 clusterstate.json file? Or is there any API available which would serve the
 purpose?

 As per the Solr wiki, we need to set the urlScheme property to https


 https://cwiki.apache.org/confluence/display/solr/Enabling+SSL#EnablingSSL-SolrCloud

 Thanks
 Hrishikesh

SolrCloud - Enable SSL

[Hrishikesh Gadre]

Hi,

If we need to enable SSL configuration for an existing Solr cluster
(hosting one or more collections), do we need to manually update the
clusterstate.json file? Or is there any API available which would serve the
purpose?

As per the Solr wiki, we need to set the urlScheme property to https

https://cwiki.apache.org/confluence/display/solr/Enabling+SSL#EnablingSSL-SolrCloud

Thanks
Hrishikesh