[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13662389#comment-13662389 ] Joel Bernstein edited comment on SOLR-4816 at 5/20/13 10:00 PM: After spending more time looking at everything that an upateRequest can do I realized that not all parts of a request are routable. The latest patch handles this by first sending all the routable updates to the correct shard. Then executing a final update request with non-routable update commands such as OPTIMIZE or deleteByQuery. This latest patch has not been tested so is for review purposes only. was (Author: joel.bernstein): After spending more time looking at everything that an upateRequest can do I realized that not all parts of a request are routable. The latest patch handles this by first sending all the routable updates to the correct shard. Then executing a final update request with non-routable update commands such OPTIMIZE or deleteByQuery. This latest patch has not been tested so is for review purposes only. Change CloudSolrServer to send updates to the correct shard --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Priority: Minor Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue changes CloudSolrServer so it routes update requests to the correct shard. This would be a nice feature to have to eliminate the document routing overhead on the Solr servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661369#comment-13661369
]
Stephen Riesenberg edited comment on SOLR-4816 at 5/18/13 2:59 PM:
---
Sorry, was away yesterday. Your patch is getting beyond my original changes, so
I won't post it again. In regards to commitWithin, you need the original
request when you create a new sub-request. Then basically
{{newRequest.setCommitWithin(originalRequest.getCommitWithin());}}
You could only do this in the new getRoutes() method if you have the original
request available. Hope that helps!
*Edit:* Also, this line:
{code:java}Slice slice =
router.getTargetSlice(doc.getFieldValue(id).toString(),doc,null,col);{code}
Would be:
{code:java}Slice slice =
router.getTargetSlice(doc.getFieldValue(id).toString(),doc,params,col);{code}
I got an NPE when passing null to the DocRouter.
was (Author: sriesenberg):
Sorry, was away yesterday. Your patch is getting beyond my original
changes, so I won't post it again. In regards to commitWithin, you need the
original request when you create a new sub-request. Then basically
{{newRequest.setCommitWithin(originalRequest.getCommitWithin());}}
You could only do this in the new getRoutes() method if you have the original
request available. Hope that helps!
Change CloudSolrServer to send updates to the correct shard
---
Key: SOLR-4816
URL: https://issues.apache.org/jira/browse/SOLR-4816
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Priority: Minor
Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
SOLR-4816.patch, SOLR-4816-sriesenberg.patch
This issue changes CloudSolrServer so it routes update requests to the
correct shard. This would be a nice feature to have to eliminate the document
routing overhead on the Solr servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13661406#comment-13661406 ] Joel Bernstein edited comment on SOLR-4816 at 5/18/13 6:01 PM: --- Stephen's comments brought up other questions which need to be decided: 1) How to handle delete by requests. 2) How to handle various action requests (commit, optimize etc...). Probably a good approach to this is to process all the direct updates first and then remove the document list from the original request and let the original request execute through the main flow. I'll work on getting this scenario into the patch. was (Author: joel.bernstein): Stephen's comments brought up other questions which need to be decided in: 1) How to handle delete by requests. 2) How to handle various action requests (commit, optimize etc...). Probably a good approach to this is to process all the direct updates first and then remove the document list from the original request and let the original request execute through the main flow. I'll work on getting this scenario into the patch. Change CloudSolrServer to send updates to the correct shard --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Priority: Minor Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue changes CloudSolrServer so it routes update requests to the correct shard. This would be a nice feature to have to eliminate the document routing overhead on the Solr servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658360#comment-13658360 ] Joel Bernstein edited comment on SOLR-4816 at 5/15/13 1:40 PM: --- No longer going to the core url directly with updates and instead sending to the baseUrl+/+collection. Also passing through the orignal params to each request. Added the getter/setter for the defaultId. Stephen, let me know if this addresses the bugs you found. was (Author: joel.bernstein): No longer going to the core url directly with updates and instead sending to the baseUrl+/+collectio. Also passing through the orignal params to each request. Added the getter/setter for the defaultId. Stephen, let me know if this addressed the bugs you found. Change CloudSolrServer to send updates to the correct shard --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Priority: Minor Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch This issue adds a directUpdate method to CloudSolrServer which routes update requests to the correct shard. This would be a nice feature to have to eliminate the document routing overhead on the Solr servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658364#comment-13658364 ] Stephen Riesenberg edited comment on SOLR-4816 at 5/15/13 1:55 PM: --- Oops, I missed your latest. I'll check it out. *Edit*: Main issue is an NPE using params (params = new ...), and missing use of the defaultCollection in params.get(). was (Author: sriesenberg): Oops, I missed your latest. I'll check it out. Change CloudSolrServer to send updates to the correct shard --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Priority: Minor Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds a directUpdate method to CloudSolrServer which routes update requests to the correct shard. This would be a nice feature to have to eliminate the document routing overhead on the Solr servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[ https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658392#comment-13658392 ] Joel Bernstein edited comment on SOLR-4816 at 5/15/13 2:21 PM: --- Added Stephen's code fixing NPE when params and/or collection is not set. was (Author: joel.bernstein): Added Stephen's code fixing NPE with when params and/or collection is not set. Change CloudSolrServer to send updates to the correct shard --- Key: SOLR-4816 URL: https://issues.apache.org/jira/browse/SOLR-4816 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 4.3 Reporter: Joel Bernstein Priority: Minor Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch, SOLR-4816-sriesenberg.patch This issue adds a directUpdate method to CloudSolrServer which routes update requests to the correct shard. This would be a nice feature to have to eliminate the document routing overhead on the Solr servers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4816) Change CloudSolrServer to send updates to the correct shard
[
https://issues.apache.org/jira/browse/SOLR-4816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13658772#comment-13658772
]
Stephen Riesenberg edited comment on SOLR-4816 at 5/15/13 8:57 PM:
---
{quote}It's collection specific.
See DocCollection.getRouter(){quote}
No javadoc on that method. In my environment, collections were set up for
implicit routing because numShards was not specified at create time. Joel
straightened us out. Only thing I found was at
http://wiki.apache.org/solr/SolrCloud#Creating_cores_via_CoreAdmin which
doesn't talk about document routing.
The NPEs I mentioned before have been fixed, one other fix was made to set the
commitWithinMS and pass params to the sub-requests *and* document router. Now I
am running a load test against this to see how it performs. I'll post an
updated patch with my changes a bit later.
{quote}We should make sure we have tests that would have caught these as
well!{quote}
I didn't run the test suite. Seems like a good idea! I'll get caught up
eventually. Having run it now with the earlier patches, seeing the NPEs. Good
stuff. Thanks.
was (Author: sriesenberg):
{quote}It's collection specific.
See DocCollection.getRouter(){quote}
No javadoc on that method. In my environment, collections were set up for
implicit routing because numShards was not specified at create time. Joel
straightened us out. Only thing I found was at
http://wiki.apache.org/solr/SolrCloud#Creating_cores_via_CoreAdmin which
doesn't talk about document routing.
The NPEs I mentioned before have been fixed, one other fix was made to set the
commitWithinMS and pass params to the sub-requests. Now I am running a load
test against this to see how it performs. I'll post an updated patch with my
changes a bit later.
{quote}We should make sure we have tests that would have caught these as
well!{quote}
I didn't run the test suite. Seems like a good idea! I'll get caught up
eventually. Having run it now with the earlier patches, seeing the NPEs. Good
stuff. Thanks.
Change CloudSolrServer to send updates to the correct shard
---
Key: SOLR-4816
URL: https://issues.apache.org/jira/browse/SOLR-4816
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Affects Versions: 4.3
Reporter: Joel Bernstein
Priority: Minor
Attachments: SOLR-4816.patch, SOLR-4816.patch, SOLR-4816.patch,
SOLR-4816.patch, SOLR-4816-sriesenberg.patch
This issue changes CloudSolrServer so it routes update requests to the
correct shard. This would be a nice feature to have to eliminate the document
routing overhead on the Solr servers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
